CSRQ Center Report on 
Education Service 
Providers 



APRIL 2006 



A 



comprehensive 
school reform 

ir^ V liMi C Kl T C D 




AMERICAN 
INSTITUTES 
FOR Research 





APRIL 2006 



CSRQ Center Report on 
Education Service 
Providers 



THE COMPREHENSIVE SCHOOL REFORM QUALITY CENTER 



The Comprehensive School Reform Quality Center 

American Institutes for Research 

1000 Thomas Jefferson Street, NW 

Washington, DC 20007 

Phone (toll-free): 866-544-8686 

E-mail: csrq@air.org 

Web site: http://www.csrq.org 



comprehensive 
school reform 




American 
Institutes 
FOR Research 



Acknowledgments 



The Comprehensive School Reform Quality (CSRQ) 
Center wishes to thank the countless people and 
organizations who made this report possible. 

Our first acknowledgment goes to the partners who 
make up the CSRQ Center: The U.S. Department of 
Educations Office of Elementary and Secondary 
Education, which funds the Center through a 
Comprehensive School Reform Quality Initiative 
grant to the American Institutes for Research (AIR). 

We are also particularly grateful to our Centers advisors 
who continue to provide us with strong guidance on 
all of our reports: Anthony Amato; Dan Goldhaber, 
Ph.D.; David Erancis, Ph.D.; Prances Harris-Burke, 
Ed.D.; Scott Joftus, Ph.D.; Will Jordan, Ph.D.; Jeff 
Valentine, Ph.D.; Jon Supovitz, Ed.D.; Katrina Kelley; 
Kenneth Wong, Ph.D.; Laura Desimone, Ph.D.; 
Margaret Raymond, Ph.D.; Matt Hornbeck, J.D.; and 
Sam Stringfield, Ph.D. 



Additionally, Geoffrey Borman, Ph.D., and Gina 
Hewes, Ph.D., provided our researchers with insight and 
knowledge to refine the work and analyze techniques 
for the highest level of rigor. Eurthermore, we wish to 
acknowledge the contributions of Rick Hess from the 
American Enterprise Institute; Renee Jacob, a former 
AIR colleague; Dean Kern, the Gharter School Program 
Director at the U.S. Department of Educations Qffice 
of Innovation and Improvement; Michael Petrilli from 
the Thomas B. Pordham Poundation; and Nancy Van 
Meter from the American Pederation of Teachers. 

We also thank our colleagues at AIR for their ongoing 
advice and expertise. 

Pinally, we wish to note that the writing of this report 
was a group effort, involving more than a dozen staff 
members of the GSRQ Genter. We thank each of them 
for their strong contributions. 



The contents of this report were developed by the Comprehensive School Reform Quality (CSRQ) Center, which is operated 
by the American Institutes for Research (AIR), under a grant from the U.S. Department of Education. The views expressed in 
this report do not necessarily reflect those of the U.S. Department of Education, AIR, or the CSRQ Center. 



About the CSRQ Center 



The Comprehensive School Reform Quality (CSRQ) 
Center is funded by the U.S. Department of Educations 
Qffice of Elementary and Secondary Education, 
through a Comprehensive School Reform Quality 
Initiative Grant, S332B030012, and is operated by the 
American Institutes for Research (AIR). 

Since 1946, AIR — one of the nations largest nonprofit 
behavioral and social science research organizations — 
has engaged in thousands of research, evaluation, 
technical assistance, consulting, and communication 
projects that help to make research relevant to policy- 
makers and practitioners. AIR’s overriding goal is to use 
the best science available to bring the most effective 
ideas and approaches to enhancing everyday life. The 
organizations work spans a wide range of substantive 
areas: education, student assessment, international 
education, individual and organizational performance, 
health research and communication, human develop- 
ment, usability design and testing, employment equity, 
and statistical and research methods. AIR conducts its 
work within a culture and philosophy of strict independ- 
ence, objectivity, and nonpartisanship. Given the variety 
of work that AIR conducts, rigorous institutional safe- 
guards have been established to guarantee that any 
potential conflict of interest is avoided. Eor additional 
information about AIR, visit http://www.air.org. 

The mission of the GSRQ Genter is to provide timely 
and reliable tools and technical assistance to support 
urban and rural educators and education decision 
makers in choosing the highest quality comprehensive 
school reform program to meet locally defined needs. 
The GSRQ Genter promises to help raise student 
achievement and improve other important student 
outcomes for millions of Americas children by helping 



education decision makers identify and apply “what 
works” in the area of comprehensive school reform. 
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Introduction 



This consumer guide for the first time ever, provides 
comparative ratings on the effectiveness and quality of 
seven widely adopted Education Service Providers 
(ESPs) — nonprofit or for-profit organizations that contract 
with new or existing public, charter, or private schools 
and/or school districts to provide comprehensive services. 
This report continues the Comprehensive School Reform 
Quality (CSRQ) Centers efforts to issue reports that help 
education decision makers sort through their options for 
whole school and district improvement. (Previous reports 
are available at http://www.csrq.org.Q 



VV | hy Is This Report Needed? 

In December 2000, a blue ribbon panel of leaders from 
more than a dozen national education organizations 
completed an intensive 9-month process to develop a 
practical tool that educators and parents could use to 
judge nationally available education assistance providers. 
The process resulted in Tool Kit for Engaging a Design- 
Based Assistance Provider: Guidelines for Ensuring the 
Quality of National Design-Based Assistance Providers 
(New American Schools, 2000). For the first time, 
these guidelines provided a framework that decision 
makers could use to measure and compare the quality 
and performance of externally developed education 
programs that claimed to provide benefits for schools 
and students. The blue ribbon panel observed that 
“schools and districts need a way to differentiate among 
the various providers and identify which design best 
fits the needs of their school” (New American Schools, 

2000, p. 2). 

In recent years, the opportunity and imperative for 
effective and flexible school improvement options have 
grown. For example, the charter school movement has 



grown significantly during the past 10 years, offering 
new ways to govern public schools and spur innovation. 
Charter schools, which operate from public funds, 
are established independent of the school district. 
Thus, charter schools have greater flexibility in day- 
to-day operations. Charters are usually held by groups 
of parents and/or teachers, academic institutions, or 
nonprofit organizations. Today, about 3,500 charter 
schools operate in more than 40 states (WestEd, n.d. a). 
(For more information on charter schools, see “About 
Charter Schools” on page 2). 

The growth of charter schools has been spurred by 
the No Child Left Behind Act (NCLB) of 2001, which 
heralded an unprecedented effort to improve school- 
ing in America. NCLB focuses on reform based on 
evidence and on accountability based on results. Each 
year since NCLB was enacted, a significant number of 
schools have failed to meet Adequate Yearly Progress 
(AYP) goals. In both the 2003-2004 and 2004-2005 
school years, approximately 25% of US. schools (or 
about 20,000 schools) did not meet AYP (Olson, 2005). 
Approximately the same number of schools is predicted 
to not meet AYP in 2005-2006. Having not met AYP, 
schools are put on notice that they must do much 
better. Additionally, increasing numbers of schools are 
beginning to enter their 5th year of not meeting AYP, 
triggering an NCLB requirement that the school enter 
a restructuring phase. Lor example, in California, 

404 schools entered this phase in 2005-2006, repre- 
senting a nearly 50% increase from the previous year 
(Center on Education Policy, 2006). Decision makers 
are likely to consider transforming these schools into 
new charter schools and/or contracting with whole 
school improvement models, such as ESPs, to manage 
the newly restructured schools. 



'Unless noted otherwise, all Web addresses displayed in this report were active as of the publishing date, April 2006. 
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About Charter Schools 

Overview 

In the 1970s, educators began to suggest that local school boards provide contracts or charters to implement new 
instructional approaches. Years later, Albert Shanker, the president of the American Federation of Teachers, built upon 
this reform soiution by recommending that entire schools be chartered to groups of educators, community organizations, 
or parents. In the late 1980s, Philadelphia implemented the idea by opening the first schools within schools, calling them 
“charter schools.” Minnesota soon foilowed and by 1991 had passed the first law that allowed the creation of charter 
schools. Since 1991, 40 states and the District of Columbia have passed similar legislation. Each state’s charter school law 
establishes a legal definition of charter schools; the process for developing a charter school; and parameters for funding, 
staffing, instruction, accountability, and admissions (WestEd, n.d. b). 

The U.S. Department of Education defines charter schools as “public schools that operate with freedom from many of the 
local and state regulations that apply to traditional public schools” (n.d. a). In practice, each state’s charter school law 
outlines the state’s approach to charter schools. Charter schools operate under a contract between the school’s founders 
and a sponsor — usually a university, local school board, district, or state. The charter outlines the school’s mission, educa- 
tional program, and methods for evaluating the school’s performance. To maintain accountability, the charter’s sponsor 
monitors a school’s adherence to the charter and holds the school accountable for students’ academic achievements and 
the school’s financial stability. 

During the past 10 years, charter schools have increasingly turned to vendors, including Education Service Providers, also 
known as ESPs, for educational services (e.g., professional development for faculty and curricular programs) and manage- 
ment services (e.g., personnel recruitment, payroll, and facilities management) (U.S. General Accounting Office, 2002). 

Additional Resources 

■ U.S. Charier Schools (http://www.uscharterschools.org) is run by a consortium of organizations: National Alliance for 
Public Charter Schools (http://www.publiccharters.org). National Association of State Directors of Special Education 
(http://www.nasdse.org), and WestEd (http://www.wested.org). According to its Web site, the consortium’s goal is to 
provide “accurate information and promising practices about and for charter schools” (WestEd, n.d. c). 

■ National Association of Charter School Authorizers (http://www.charterauthorizers.org) is a “nonprofit membership association 
of educational agencies across the country that authorize and oversee public charter schools” (National Association 
of Charter School Authorizers, n.d.). Its Web site offers information and services to authorizers and others who are 
interested in the charter school movement. 

■ U.S. Department of Education’s Public Charter Schools Program (http://www.ed.gov/programs/charter/index.html) “supports 
the planning, development, and initial implementation of charter schools” (U.S. Department of Education, n.d. b). 



People who are responsible for improving these schools 
seek to implement improvement approaches supported 
by scientifically based research. But education decision 
makers face an increasing number of whole school 
improvement options that claim to offer research-based, 



research-proven services. These options include ESPs, 
the subject of this consumer guide. 

For the purposes of this report, the CSRQ Center 
defines ESPs as nonprofit or for-profit organizations 
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that contract with new or existing public, charter, or 
private schools and/or school districts to provide 
comprehensive services to schools, including, but not 
limited to, educational programming and administrative 
services. Educational programming includes curriculum 
design, professional development, and student assess- 
ment tools. Administrative services include, but are not 
limited to, operation management services (student 
enrollment, school marketing), financial management 
services (payroll assistance, budget oversight), facilities 
management services (maintenance and use of facili- 
ties), and human resources management (hiring and 
training staff, staff benefits). Many of the comprehen- 
sive services provided by ESPs are comparable to those 
offered by other whole school improvement providers, 
such as comprehensive school reform (CSR) models 

According to Profiles of For-Profit Education 
Management Organizations 2004-2005, Seventh Annual 
Report (Molnar, Garcia, Sullivan, McEvoy, & Joanou, 
2005), 59 for-profit education service providers 
manage 535 schools in 24 states and the District of 
Columbia. Of these schools, 86% are charter schools. 
The seven ESPs reviewed in this report currently work 
in more than 350 public schools — including many 
charter schools. 

The growth of ESPs as an option for whole school 
improvement, when added to the choices available 
through the hundreds of comprehensive school reform 
model providers, means that decision makers face a 
bewildering set of choices, with little guidance to help 
sort through options. Consumers still have too few 
independent and credible sources to turn to when 
making important adoption decisions (Shaul, 2002). 

This report serves as a consumer guide that will help 
decision makers sort through claims about which 
approaches could truly meet the needs of students. It 
is the first comprehensive review of ESPs ever issued. 
To prepare this report, the CSRQ Center screened 
nearly 940 documents and reviewed about 40 studies 
on seven widely implemented ESP models. We used 



rigorous standards that are aligned with the require- 
ments for scientifically based research established by 
NCLB. Each model is rated on a number of dimen- 
sions, including evidence of raising student achieve- 
ment. The reviews of the individual models are written 
to provide education decision makers with profiles of 
each model and the evidence that they need to make 
decisions to meet locally defined needs. 



w hat Is Whole School Improvement and 
Why Does It Matter? 

Eor more than two decades, the school-level adoption 
and effective implementation of externally developed 
and research-based whole school improvement models 
have been used increasingly to raise student achieve- 
ment. These models have been tried in hundreds of 
schools nationwide, most of which are high poverty 
and low performing. This trend is driven by the recog- 
nition that school improvement efforts are complex 
and require a coordinated, systematic approach that 
addresses every aspect of a school, including curricu- 
lum, instruction, governance, scheduling, professional 
development, assessment, and family and community 
involvement. Rather than use individual, piecemeal 
programs or approaches, effective whole school 
improvement models integrate research-based practices 
into one unified effort to raise student achievement 
and achieve other important outcomes, such as reduced 
dropout rates or improved behavior. 

Many schools that adopt the whole school improve- 
ment approach choose an external model to provide 
a research-based, replicable set of practices. These 
external models, which are offered by a variety of 
service providers, are meant to be blueprints to help 
a school make improvements in a number of areas. 
Although their foci, philosophies, and methods vary, 
these models are designed based on research and are 
intended to help the school raise student achievement. 
To support implementation, whole school improvement 
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models typically provide schools with materials, pro- 
fessional development, and technical assistance. Other 
schools that adopt a whole school improvement 
approach may choose to develop their own improve- 
ment models, putting together research-hased elements. 

The whole school improvement approach has evolved 
from more than two decades of systematic improvement 
efforts based on the adoption of external schoolwide 
improvement models. This trend accelerated in the early 
1990s, when, after decades of concentrating on pro- 
grams targeted at individual students at risk of academic 
failure, a new idea based on a comprehensive approach 
to school improvement was conceived. The RAND 
Corporation published Federal Policy Options for 
Improving the Education of Low-Income Students, 
Volume I, Findings and Recommendations in 1993. This 
report suggested to the federal government that to reap 
the biggest impact, funds from Title I (previously called 
Chapter I) would be best spent on schoolwide improve- 
ment (Rotberg & Harvey, 1993). These ideas were soon 
incorporated into the Title I program. At about the 
same time. New American Schools began to operate as 
an advocate for whole school improvement and a sup- 
porter of the development of high-quality whole school 
improvement models (Stringfield, Ross, & Smith, 1996). 

The whole school improvement approach gained fur- 
ther momentum with the 1997 passage of the federal 
Comprehensive School Reform Demonstration pro- 
gram. Through this program. Congress provided 
dedicated funding to support the adoption of whole 
school improvement strategies throughout the country. 
The 2001 Elementary and Secondary Education Act, 
also known as NCLB, gave further momentum to the 
whole school improvement approach by changing it 
from a demonstration project to a full-fledged federal 
program called the Comprehensive School Reform 
Program. According to NCLB, whole school improve- 
ment models must be scientifically based. This means 
that a model or approach must demonstrate strong 
research evidence that it can improve students’ academic 



achievement. Today, regardless of the funding source, 
the use of schoolwide improvement models is likely to 
remain an important strategy for improving schools, 
particularly those that fail to make AYR 

Whole school improvement models are of great interest 
to decision makers because of claims from the model 
provider that their models are research-based and 
provide the training and other supports needed to 
encourage a coordinated approach to achieve student 
success. The research evidence to date indicates that 
some whole school models are more effective than 
others and those results vary greatly — even for effective 
models — depending on the quality of implementation 
(see Desimone, 2000). 



ow Can Educators Meet the Challenge 
of Evidence-Based Decision Making? 

Critics often claim that decisions in the education field 
are driven by whims and fads, thoughtlessly adopted 
and easily abandoned. Although this is an exaggeration, 
it is nevertheless true that despite billions of dollars and 
countless hours of well-intentioned efforts, educators 
and policymakers still cannot say, with confidence, how 
best to bring about the many desired improvements. 
Better research and evidence, when combined with 
sound professional judgment, can help guide the way 
toward solid and sustained improvement. However, 
educators, policymakers, and the public cannot be 
expected to do what works until they actually know 
what works. 

The education community increasingly turns to 
research to help sort through its school improvement 
options. This reliance on research helps to satisfy 
NCLB’s requirement that school improvement efforts 
be driven by scientifically based research. More 
importantly, however, it helps to meet the urgently felt 
need on the part of educators and policymakers to 
ensure that their efforts improve the lives of children. 
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Researcher Tom Corcoran (2003) points out some of 
the challenges in transforming education into an 
evidence-hased field. In a study conducted in three 
districts, he found that 

School district leaders want to make 
evidence-hased decisions and they are 
making efforts to huild evidence-hased 
cultures in their central offices and schools. 

But, significant progress is being hampered 
hy the inadequacy and confusion of the 
existing research, its availability to school 
and district-level staff, and reliance by staff 
on decision-making patterns that focus on 
philosophy rather than effects, (p. 1) 

In addition to the challenges confronted by districts, 
education stakeholders — including teachers, adminis- 
trators, policymakers, and state- and district-based 
evaluators — are hard pressed to keep up with the 
volume of approaches and initiatives that must be 
studied. One recent nationwide review of education 
program evaluation efforts at the state level (Raymond, 
Bortnik, & Gould, 2004) found that 

Most states infrequently evaluate their pro- 
grams, if at all . . . [A] bout a third of states 
do practically none, another third does a 
little, and a third does a noticeable number 
of evaluation studies . . . [L]ess than 10% 
of all the studies purporting to be impact 
evaluations used random assignment or 
quasi-experimental designs, (pp. viii-ix) 

In short, few evaluation studies are conducted, and 
even fewer studies are rigorous enough to provide 
reliable findings. In addition, the researchers found 
that even the results of these infrequent and flawed 
evaluations were disseminated only sporadically, thus 
providing little guidance to decision makers. 



A further impediment to building evidence-based 
practice and policy in education is the lack of research 
studies and findings that provide practical guidance. 
Many studies in education do not focus on questions 
that are critical to decision makers, such as what works, 
under what circumstances, and for which students. Also, 
some of the research that could potentially act as a guide 
is very hard to access or understand. Thus, solid research 
evidence is often undervalued or ignored (Huang, Reiser, 
Parker, Muniec, & Salvucci, 2003; Sutton & Thompson, 
2001). As a result, when educators seek and demand 
evidence to help answer their questions, they are left 
either disappointed by the lack of relevant research or 
are challenged to make meaning out of the findings. 

Even when educators and decision makers have com- 
mitted to the adoption of models that have track records 
of effectiveness, they are often challenged to find, 
interpret, and apply relevant research. The selection 
process is also challenging, because interpretations of 
findings across evaluation studies of the same or similar 
models are difficult to make due to variations in imple- 
mentation, characteristics of participating students, 
rigor of the research design, and other factors. 

Fortunately, a number of efforts are underway to 
improve the value of research for education decision 
makers. Many of these efforts are sponsored by the 
U.S. Department of Education and seek to (a) improve 
the quantity and quality of education research, (b) make 
it more relevant to educators, and (c) ensure that it is 
available in a timely manner and in easily accessible 
formats and language. For example, the U.S. Department 
of Education and others have issued guidance on 
judging the quality and relevance of research findings 
(see “Resources for Judging Research in Education” on 
page 6).^ Furthermore, the What Works Clearinghouse 
(WWC) — sponsored and managed by the Institute of 
Education Sciences of the U.S. Department of 
Education — provides educators, policymakers. 



^The CSRQ Center provides further guidance on this topic on pages 6-8 of Works in Progress: A Report on Middle and High School Improvement Programs 
(CSRQ Center, 2005). 
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Resources forjudging Research in Education 

Fashola, 0. S. (2004). Being an informed consumer of quantitafive educational research. Phi Delta Kappa, 85, 532-538. 

This article inciudes a user-friendly description of the nature of scientific research. Specific guideiines are offered on how 
to evaluate the quality of an evaluation study and how to reiate findings to the educator’s own school or district context. 

Fleischman, S. (2005). Research matters: Moving to evidence-hased practice. Educational Leadership, 63, 87-90. 

This coiumn outlines concerns that educators have expressed regarding access to research and their abiiity to apply this 
research, it aiso provides resources that can help educators bridge the gap between research and practice. 

Lauer, P. A. (2004). A poiicymaker’s primer on education research: How to understand, evaluate and use it. Aurora, CO: Mid- 
Continent Research for Education and Learning, Denver, CO: Education Commission of the States. Retrieved December 1, 2004, 
from http://www.ecs.org/html/educationlssues/Research/primer/foreword.asp 

This primer addresses how to determine the trustworthiness of research and whether research warrants policy changes. 
It also includes a statistics tutorial and a glossary. 

Slavin, R. E. (2003). A reader’s guide to scientifically based research. Educational Leadership, 60, 12-16. 

This article presents a review of criteria to use when selecting scientific research to review and how to evaluate the quality 
of the research. 

Stringfield, S. (1998, Fall). Choosing success. American Educator. Retrieved December 1, 2004, from http://www.aft.org/ 
pubs-reports/american_educator/fall98/ChoosingSuccess.pdf 

This is a practical guide on how to select a model using such criteria such as model goals, research base, and associated 
costs. 

U.S. Department of Education, Institute of Education Sciences. (2003). Identifying and impiementing educationat practices 
supported hy rigorous evidence: A user triendfy guide. Washington, DC: Author. Retrieved December 1, 2004, from 
http://www.excelgov.Org/usermedia/images/uploads/PDFs/User-Friendly_Guide_12.2.03.pdf 

This publication points out the importance of using rigorous evidence and provides guidance when applying it to make 
decisions about program and model adoption. 

U.S. Department of Education, Institute of Education Sciences. (2003). Random assignment in program evaluation and intervention 
research: Questions and answers. Washington, DC: Author. Retrieved Dctober 10, 2005, from http://www.ed.gov/rschstat/eval/ 
resources/randomqa.html 

This brochure explains the nuts and bolts of why and how random assignment evaluations are conducted and answers 
some frequently asked questions. 



researchers, and the public with a central, trusted 
source of scientific evidence of what works in educa- 
tion. WWC systematically searches for, evaluates, and 
reports on the evidence of effectiveness of programs, 
products, practices, and policies that claim to improve 
student outcomes. Throughout the coming years, 
WWC will review many topics of interest to education 
decision makers, including programs to raise math 



and reading achievement, reduce dropout rates, and 
improve character education. WWC’s reports are 
available at http://www.whatworks.ed.gov. 

Sorting through and making sense of research is hard 
work, even for research scientists with years of train- 
ing and experience. Despite substantial advances in 
developing standards and processes forjudging and 
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The CSRQ Center’s Model Registry 

Reports from the CSRQ Center can review only a limited number of whole school improvement models. Some education 
decision makers may be interested in additional models, including new or smaller ones that have not yet been reviewed 
in reports by the CSRQ Center. Thus, the CSRQ Center launched a Model Registry in fall 2005 so that service providers 
have the opportunity to share nonevaluative information about models not included in reports from the CSRQ Center. 

The Model Registry is nonevaluative, and any provider who wishes to register information on a CSR or ESP model may 
do so. Users should be aware that each model provider has supplied the information in this Registry. The CSRQ Center will 
conduct a minimal amount of fact checking for each model. The Model Registry provides basic background information 
for each whole school improvement model: 

■ Focus and mission of the model 

■ Grade levels that the model serves 

■ Subject areas that the model covers 

■ Descriptions and citations of research that demonstrates the model’s effectiveness on student achievement and 
other outcomes 

■ Descriptions of the link between research and the model’s design 

■ Description of the model’s services and supports to schools 

■ Cost of the model 

Providers that would like to submit information about their models can register on the CSRQ Center’s Web site: 
http://www.csrq.org/CSRProgramRegistry.asp. 



adding up the evidence in education, researchers often 
disagree. Although procedures exist for reviewing and 
comparing a large number of studies, the process is 
often complex and painstaking. Therefore, education 
decision makers often turn to others to sort through 
the evidence and report it as actionable information. 



H ow Can Education Decision Makers 
Use This Report? 

This report provides education stakeholders with a 
decision-making tool to help them sort out options 
from ESPs that are available to meet local needs. The 
ratings that are applied to the seven models in this 



report are intended to clarify options, not to point to 
or endorse best buys. Together, these models represent 
a significant portion of the total number of ESPs being 
used by schools. Each model included in this report 
serves more than 20 schools in at least three states and 
is available for adoption in almost all states. (Eor a 
detailed discussion about this report, see “About This 
Report” and “Methodology”) 

Although this report reviews evidence on widely 
adopted models, it does not represent an evaluation 
of the ESP improvement strategy as a whole. To satisfy 
the interest expressed by many stakeholders in knowing 
about as many whole school improvement models as 
possible, the CSRQ Center’s Web site provides a Model 
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Registry that allows any whole school improvement 
model provider to enter information about its model 
(see “The CSRQ Centers Model Registry” on page 7). 
In addition, we believe that the review framework 
described in “About This Report” can be used by 
education consumers to ask probing questions of each 
model being considered, even if the model is not includ- 
ed in one of our reports. For example, consumers can 
ask model providers to provide them with rigorous 
research evidence on effectiveness and to demonstrate 
how this evidence aligns with the standards set by the 
CSRQ Center. 

Finally, readers should be aware that a variety of organ- 
izations provides publications, tools, and guidance to 
help educators and others who are considering the 
adoption and effective implementation of whole school 
improvement models. The CSRQ Center’s Web site 
(http://www.csrq.org) provides a list of helpful organi- 
zations and resources. 



R. eferences 

Center on Education Policy. (2006). Wrestling the devil 
in the details: An early look at restructuring in 
California. Washington, DC: Author. 

Comprehensive School Reform Quality Center. (2005, 
January). Works in progress: A report on middle and 
high school improvement programs. Washington, 
DC: Author. Retrieved Qctober 26, 2005, from 
http://www.csrq.org/worksinprogress.asp 

Corcoran, T. (2003, November). The use of research 
evidence in instructional improvement (CPRE 
Policy Briefs, RB-40). Philadelphia: Graduate 
School of Education, Consortium for Policy 
Research in Education, University of Pennsylvania. 

Desimone, L. (2000). Making comprehensive reform 
work (Urban Diversity Series 112). New York: 



ERIC Clearinghouse on Urban Education. (ERIC 
Document Reproduction Service No. ED 441915) 

Huang, G., Reiser, M., Parker, A., Muniec, J., & 

Salvucci, S. (2003). Institute of Education Sciences: 
Findings from interviews with education policy- 
makers. Washington, DG: US. Department of 
Education. 

Molnar, A, Garcia, D., Sullivan, G., McEvoy, B., & 
Joanou, J. (2005, April). Profiles of for-profit 
education management organizations 2004-2005, 
seventh annual report. Tempe: Education Policy 
Studies Laboratory, Arizona State University. 

National Association of Gharter School Authorizers. 
(n.d.). Retrieved March, 16, 2006, from 
http://www.charterauthorizers.org/site/nacsa/ 
section.php?id=l 

New American Schools. (2000). Tool kit for engaging 
a design-based assistance provider: Guidelines for 
ensuring the quality of national design-based 
assistance providers. Arlington, VA: Author. 

Qlson, L. (2005, September 7). Defying predictions, 
state trends prove mixed on schools making 
NCLB targets. Education Week, pp. 1, 26-27. 

Rotberg, L, & Harvey, J. (1993). Federal policy options 
for improving the education of low-income students, 
volume I, findings and recommendations. Santa 
Monica, GA: RAND Gorporation. 

Raymond, M. E., Bortnik, K., & Gould, R. (2004, 
April). Program evaluation capacity in state 
departments of education. Stanford, GA: Hoover 
Institution, Genter for Research on Education 
Qutcomes, Stanford University. Retrieved 
Qctober 26, 2005, from http://credo.stanford.edu/ 
downloadsZori.pdf 

Shaul, M. S. (2002). Insufficient research to determine 
effectiveness of selected private education 



HOW CAN EDUCATION DECISION MAKERS USE THIS REPORT 



8 




INTRODUCTION 



companies (GAO Publication No. 03-11). 
Washington, DC: U.S. General Accounting Office. 

Stringfield, S., Ross, S., & Smith, L. (Eds.). (1996). 

Bold plans for school restructuring. Mahwah, NJ: 
Erlbaum. 

Sutton, S. M., & Thompson, E. (2001, Winter). An in- 
depth interview study of health care professionals 
and their research needs. Social Marketing 
Quarterly, VII{4). 

U.S. Department of Education, (n.d. a). Find schools 
& after-care: School choices for parents. Retrieved 
March 16, 2006, from http://www.ed.gov/ 
parents/schools/ choice/definitions.html#cs 

U.S. Department of Education, (n.d. b). Public Charter 
Schools Program. Retrieved March 16, 2006, 



from http://www.ed.gov/programs/charter/ 
index.html 

U.S. General Accounting Office. (2002). Public schools: 
Insufficient research to determine effectiveness of 
selected private education companies. Washington, 
DC: Author. 

WestEd. (n.d. a). State by state #’s. Retrieved March 16, 
2006, from http://www.uscharterschools.org/cs/ 
sp/query/q/1595 

WestEd. (n.d. b). U.S. Charter Schools. Retrieved March 
16, 2006, from http://www.uscharterschools.org 

WestEd. (n.d. c). U.S. Charter Schools. Retrieved March, 
16, 2006, from http://www.uscharterschools.org/ 
pub/uscs_docs/blk/about.htm 



REFERENCES 



9 



About This Report 



This section provides readers with general information 
on how the Comprehensive School Reform Quality 
(CSRQ) Center reviewed models on five categories of 
quality and effectiveness, including a description of the 
process to establish the rating system, an overview of 
the rating process, and an explanation of the ratings. In 
keeping with the consumer orientation of the report, we 
have tried to limit our use of technical jargon and to 
provide clear, straightforward discussions of method- 
ological issues involved in conducting the reviews. 
“Methodology,” which follows, reviews the scientific 
procedures that were followed to produce this report. 



M ow Are Models Rated by This Report? 

This CSRQ Center report provides a series of reviews 
of seven widely adopted models from Education 
Service Providers (ESPs). As a group, they operate in 
about 350 schools, likely representing 60-65% of all 
the schools currently being served by ESPs. (Eor more 
information on ESPs, see “What Is an Education 
Service Provider?”) 

Although summaries of overall evidence of effective- 
ness and quality are crucial to solid decision making, 
they can also be misleading. Eor example, researchers 
have frequently noted that most models vary in their 
effectiveness from school to school. That is, in some 
schools they work well and in others hardly at all 
(Borman, Hewes, Qverman, & Brown, 2002). Qften 
these variations in model effectiveness are about as 
large as the variation in effectiveness from one model 
to another. Thus, decision makers should keep in mind 
that even those models that received lower ratings in 
this report may be good options in certain circum- 
stances. Eor instance, because implementation is such 
an important variable in ensuring good results, it may 
be better for a school or district to adopt a model that 



What Is an Education Service Provider? 

For the purposes of this report, ESP is defined as a 
nonprofit or for-profit organization that contracts with 
new or existing public, charter, or private schools 
and/or school districts to provide comprehensive services 
to schools. Including, but not limited to, educational 
programming and administrative services. Educational 
programming includes curriculum design, professional 
development, and student assessment tools. 
Administrative services include, but are not limited to, 
operation management services (student enrollment, 
school marketing), financial management services 
(payroll assistance, budget oversight), facilities man- 
agement services (maintenance and use of facilities), 
and human resources management (hiring and training 
staff, staff benefits). The comprehensive services pro- 
vided by ESPs are comparable to services provided by 
other whole school Improvement models. 



might meet the needs of the local leadership and school 
community, despite a lower rating. Alternatively, if a 
district or school commits to doing the needed work 
to ensure that its chosen model is implemented prop- 
erly, it may wish to adopt a higher rated model even 
if it may encounter some resistance. 

As with all consumer choices, decision makers must 
weigh the pros and cons of their model selection. This 
report is not intended to dictate decisions or pick 
winners and losers, but rather it aims to clarify choices 
by providing the most rigorous evidence and user- 
friendly information to date on the available options 
to meet local school improvement needs. 

Each review first offers basic information on the ESP 
model, including the model’s mission and focus, year 
introduced in schools, grade levels served, number of 
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schools served, and costs. In particular, we tried to 
gather as much detailed information as possible about 
the costs of adopting and implementing each model, 
because this is a key consideration for schools and 
districts. Unfortunately, models do not uniformly 
report this information, and costs vary widely. Ideally, 
for each model, we would have provided an estimated 
total cost of implementation, which would have 
included the services and materials provided by the 
model and any additional labor or materials expenses 
(e.g., new textbooks or software or release time for 
teacher professional development or common plan- 
ning). Each review provides as much information as 
we were able to gather from the provider and from 
publicly available sources. As consumers, schools and 
districts are in a strong position to (a) require each 
model to specify all of its expected costs in comparable 
formats and (b) estimate the budgetary impact of local 
changes that might have to be made to successfully 
implement the model. We urge consumers to engage the 
models in this discussion early in the adoption process. 

The Tool Kit for Engaging a Design-Based Assistance 
Provider: Guidelines for Ensuring the Quality of National 
Design-Based Assistance Providers, issued in 2000 by a 
blue ribbon panel of education stakeholders, established 
a set of standards to which all model developers should 
be held (New American Schools, 2000). The CSRQ 
Center, including our advisory committee, used these 
standards and its experience working in the whole 
school reform field for the past decade to develop a set 
of measures to rate the quality and effectiveness of 
ESP models. Without a doubt, academic outcomes are 
a critical measure of a model’s performance. Educators, 
administrators, policymakers, and the public all want 
to know: Will the model we are considering for our 
school improve our students’ academic performance? 
In addition, decision makers want evidence in other 
critical areas that assures them that a model will pro- 
vide not only help to improve student achievement 
but also deliver services that are considered important, 
such as providing support for model implementation 



or for effective parental and community involvement. 
Therefore, this report evaluates evidence on five cate- 
gories for each model. 

Category 1 : Evidence of Positive Effects on Student 
Achievement 

A school or district considering implementing a model 
should conduct a self-assessment to identify its own 
strengths and weaknesses and to seek a model that 
will help it address these areas. As part of this process, 
consumers need to know whether a service provider can 
help their schools raise achievement levels of specific 
student groups and whether a model can demonstrate 
positive impacts on student achievement in specific 
subject areas. Category 1 examines the extent to which 
a model can demonstrate, using research of reasonable 
quality, a positive impact on student achievement. 

This category is comprised of three subcategories. 

Subcategory la focuses on a model’s evidence of 
positive overall effects on student achievement. The 

rubrics in this subcategory establish standards by which 
research on a model’s overall impact on student out- 
comes is evaluated. This may be the only subcategory 
that matters for many consumers. However, decision 
makers should consider that our review of nearly 
40 studies on these models revealed only an emerging 
evidence base on the effectiveness of individual ESPs. 
Thus, some models in our review may have received a 
relatively low rating based on the current small research 
base of studies demonstrating effectiveness. This means 
that while many models may be able to consistently 
improve student outcomes, such capacity may not yet 
be based on rigorous research evidence. In time, many 
models may and should be able to provide greater evi- 
dence of positive impact on student achievement. We 
recommend that consumers decide which models they 
will consider based on (a) the CSRQ Center’s ratings 
in all categories and (b) a careful review of the detailed 
profile provided for each model. 
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Subcategory lb examines whether a model can 
demonstrate evidence of positive effects for diverse 
student populations. Readers should note that many 
schools implementing the seven models reviewed in 
this report are high-poverty schools. Although we were 
not able to gather the information on the percentage 
of Title I students served by these models, federally 
funded whole school reform models on average serve 
school populations with a poverty rate of about 70% 
(Southwest Educational Development Laboratory, n.d.). 
Therefore, even when a model does not break out its 
results by specific subpopulations, it can be assumed 
that overall these studies measure impact in highly 
challenging circumstances. The models that reported 
outcomes for specific student populations should be 
commended for their efforts to provide consumers with 
this additional disaggregated information, which is 
rarely available. Therefore, even in instances in which a 
model provided evidence that was rated on the low end 
of our rating scale, readers should consider that other 
models have not reported this evidence and therefore 
provide less information on which to make a decision. 

Subcategory Ic examines whether a model can demon- 
strate evidence of positive effects for specific subject 
areas. Similar to subcategory lb, few models provided 
evidence of their impact in specific subject areas. When 
we were able to find this evidence for specific subjects, 
the most common content areas were reading or math. 
Therefore, even in instances in which a model provided 
evidence that was rated on the low end of our rating 
scale, consumers should consider that other models 
have not reported this evidence and therefore provide 
less information on which to make a decision. 

Category 2: Evidence of Positive Effects on Additional 
Outcomes 

Category 2 was developed to provide consumers with 
information about a model’s effects beyond student 
achievement. Although student achievement is usually 
the outcome of primary concern to those seeking tools 



to improve their schools, consumers also want to know 
whether a model can help a school improve additional 
nonachievement outcomes, such as student discipline, 
student attendance, school climate, retention/promotion 
rates, and teacher satisfaction. However, our attempts 
to rate models in these areas faced two key challenges. 
First, the amount of available evidence in these areas 
is insufficient to adequately judge the quality of most 
models. Second, currently available measurement 
tools for these areas are much less reliable and sound 
than the CSRQ Center would prefer. For example, 
although steps are now being taken to remedy this 
situation, student attendance is measured differently 
across schools and districts. The additional outcomes 
covered in Category 2 are the outcomes that were 
most commonly examined in the research literature 
across models. 

Consumers must make a distinction between models 
that specifically claim to help schools improve in the 
areas outside of student achievement and those that do 
not. For example, some models include components 
that are designed specifically to help improve student 
discipline, while other models do not. Improvement in 
student discipline may be a side effect of implementing 
a given model — even if that model does not claim, or 
was not developed, to improve that particular outcome. 
However, if a model promises that it can help a school 
to improve student discipline, then that model should 
be able to demonstrate that it can deliver on its promise. 
Consumers should proceed with caution if a model was 
developed to help schools improve in a specific area 
but cannot provide solid evidence of effectiveness. 

Category 3: Evidence of Positive Effects on Parent, 
Family, and Community Involvement 

The CSRQ Center’s audiences have indicated that 
consumers also want to know whether a model can help 
a school improve its level of family and community 
involvement. Research also suggests that high perform- 
ing schools may benefit from having strong family and 
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community involvement. Moreover, citizens in every 
community have a right and a responsibility to be 
engaged in improving schools for their children and 
for society at large. Family and community involvement 
in reform efforts can spur and may help sustain long- 
term improvements. Based on this information, the 
CSRQ Center developed rubrics to determine whether 
a model can demonstrate that it helps schools improve 
family and community involvement. Consumers 
should keep in mind that some models, while acknowl- 
edging the desirability of parental involvement in 
schooling, do not count on parental involvement to 
deliver improved student achievement. Decision 
makers should note this as they review models that 
may have higher ratings on student outcomes and 
lower ones on family and community involvement. 
Some ESPs have decided to focus on strengthening 
elements other than community involvement to 
achieve their stated outcomes. 

For Categories 1 (student achievement), 2 (other edu- 
cational outcomes), and 3 (family and community 
outcomes), we synthesized quantitative evidence gath- 
ered through the review of existing research articles 
on the models reviewed. Whenever possible, we have 
provided information on model results for specific 
student groups or specific types of school settings. 

Category 4: Evidence of a Link Between Research and 
the Model’s Design 

As schools and districts increasingly heed the national 
call to implement scientifically based reform, consumers 
wiU need to know whether a model can clearly demon- 
strate links between research and the components of 
its design. 

Clear explanations from ESPs on model design can 
help school staff understand the model and accept 
changes that they will be required to make. In addition, 
consumers who may be considering a newer model 
with lower evidence of effectiveness must consider 



whether the model’s design is based on solid research. 
A newer model may not have had sufficient time to 
conduct enough research on its effectiveness, but that 
model should be able to clearly demonstrate that it 
can work; that is, that the model’s design is based on 
solid evidence of what works. Of course, over time a 
model must demonstrate that it does work. The ratings 
for Category 4 measure how clearly and explicitly the 
materials reviewed by the CSRQ Center demonstrate 
links between research and the model’s design. 
Through phone conversations with the model’s direc- 
tor, conversations with a group of randomly selected 
districts or charter authorizers for each model, and 
a review of model materials, we rated whether the 
model has linked its components — such as organiza- 
tion and governance, professional development, and 
technology — to a literature base. Consumers should 
be aware that it was beyond the scope of this report 
to review whether the research cited by the models 
is itself highly rigorous. Other researchers and organi- 
zations, such as the What Works Clearinghouse, help 
address this issue. 

Category 5: Evidence of Services and Support to 
Schools to Enable Successful Implementation 

Even the most well-designed, well-researched models 
can fail to produce positive results if implemented 
poorly. Implementing any model requires schools 
and districts to expend significant amounts of money, 
time, and effort over a long period of time. If con- 
sumers are going to make this kind of investment, 
they need to feel confident that the model provider 
can offer adequate, high-quality services and supports 
to help school staff fully and faithfully implement the 
model. The CSRQ Center created Category 5 to rate 
two subcategories: a model’s readiness to be imple- 
mented successfully and the quality of professional 
development and technical assistance that the model 
provides to schools. 
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Subcategory 5a reviews the model’s evidence of 
readiness for successful implementation. Under this 
subcategory, we assessed the following dimensions: 

■ Provider tracks and supports full implementation 
in schools. 

■ Provider helps schools allocate resources needed to 
fully implement the ESP model. 

Subcategory 5b reviews the model’s evidence of pro- 
fessional development/technical assistance for suc- 
cessful implementation. Under this subcategory, we 
assessed the following dimensions: 

■ Provider offers comprehensive training opportuni- 
ties and supporting materials. 

■ Provider ensures that professional development 
effectively supports full model implementation. 

■ Provider develops a school’s internal capacity to 
provide professional development. 

For Categories 4 (link between research and the model’s 
design) and 5 (professional development and technical 
assistance), we synthesized and reported qualitative 
data gathered through phone conversations with model 
directors and up to three districts or charter authorizers 
and reviewed publicly available documentation on the 
models under review. These two categories rate the 
effectiveness of the ESP’s delivery of services to schools. 

Decision makers and consumers need to know that 
the model they adopt is effective and that its services 
will be delivered effectively. As readers will note, many 
of the models reviewed in this report take from 3 to 
5 years to fully implement and demonstrate results. 
Consumers must have confidence that the ESPs which 
they engage are financially sound organizations that 
will be able to deliver high-quality services over the 
life of the contract. To date, no one has reviewed this 
type of critical consumer information. The CSRQ 
Center has worked with financial and organizational 



experts to develop a set of standards that will permit 
consumers to make more informed and confident 
long-term commitments. 



ow Was the Rating System Developed 
and Applied? 

The production of this report was guided by the CSRQ 
Center’s Quality Review Tool (QRT). The QRT provides 
the criteria for independent, fair, and credible model 
reviews. (Greater detail regarding the methods used in 
this study is available in “Methodology.”) To ensure 
that the QRT is valid, reliable, credible, and useful, the 
QRT development process involved several steps. First, 
staff developed review frameworks in consultation 
with some of the nation’s most respected education 
researchers, model evaluators, and school improvement 
experts. Then, the QRT was reviewed and revised with 
the help of the CSRQ Center’s Advisory Committee, 
a nationally respected panel of experts that includes 
leading education practitioners, methodologists, and 
researchers from a variety of fields, including education, 
sociology, psychology, and economics (see Table 1). 
Finally, the QRT drew on prior and current efforts to 
conduct rigorous research reviews — including Herman 
et al. (1999) and Borman et al. (2002) — and standards 
set by the What Works Clearinghouse. 

The forms, rubrics, and evaluation criteria that are 
part of the QRT have been carefully designed to guide 
the CSRQ Center’s reviews of reform models. The 
tools are intended to make the review process clear, 
transparent, and rigorous. The QRT review process is 
divided into three parts. Each part guides a distinct 
phase of the review process. Figure 1 depicts the QRT 
research review and reporting process. 

QRT Part 1 is an information cataloguing system that 
allowed the research team to acquire as much informa- 
tion as possible about all models being reviewed. It 
consisted of a multifaceted process for collecting and 
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Table 1 . The CSRQ Center’s Advisors 
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President 
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Jeff Valentine, Ph.D. 


Professor 


Duke University, Department of Psychology 


Jon Supovitz, Ed.D. 


Research Assistant Professor 


University of Pennsylvania, Graduate School of 
Education and Senior Researcher, Consortium for 
Policy Research in Education 


Katrina Kelley 


Director, Council of Urban Boards 
of Education 


National School Boards Association 


Kenneth Wong, Ph.D. 


Professor of Education Policy and 
Director, Urban Education Policy 
Master’s Program 


Brown University, Education Department 
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and Education 


Vanderbilt University, Peabody College of Education 


Margaret Raymond, Ph.D. 
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and Director, Center for Research 
on Education Outcomes 


Stanford University, Center for Research on 
Education Outcomes 


Matt Flornbeck, J.D. 


Principal 


Hempstead Elementary School, Baltimore, Maryland 


Sam Stringfield, Ph.D. 


Distinguished University Scholar 


University of Louisville, College of Education and 
Human Development 


Scott Joftus, Ph.D. 


President 


Cross and Joftus, LLC 


Will Jordan, Ph.D. 


Associate Professor 


Temple University, College of Education 



verifying information from literature reviews, contacts 
with model staff, and conversations with districts 
implementing the model or charter authorizers that 
review and approve the charters. QRT Part 1 included 

■ Gathering publicly available materials about the ESP 
models from academic and educational journals, 
the Internet, and the model developers themselves; 

■ Reviewing the materials to develop an initial 
description of the ESP model; 



■ Contacting the ESP to confirm the description and 
to request the following information: studies of the 
model’s implementation and effectiveness, model 
benchmarks, and the research base for the model’s 
design; and 

■ Holding conversations with three districts or charter 
authorizers for each model (chosen at random) to 
verify the descriptive information and better 
understand the implementation process. 
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QRT1 



Figure 1 . QRT Process 



QRT 2 



QRT 3 




O Parti O Part 2 Q Parts 



QRT Part 2 helped to analyze the model’s evidence of 
effectiveness and research base. It examined the rigor 
of the research design of each individual study on an 
ESP model’s effectiveness. QRT Part 2 did not examine 
the strength of an ESP model’s impact. Instead, it 
judged the quality of the research design supporting 
its evidence of impact. QRT Part 2 included 

■ Determining which studies met the CSRQ Center’s 
standards for causal validity of the outcome meas- 
ures, collecting contextual and statistical informa- 
tion about each study; and 

■ Rating the rigor of the research design and identi- 
fying the studies of sufficient quality to be included 
in Part 3 of the review. 

QRT Part 3 applied rubrics that established standards 
against which evidence of a model’s impact could be 
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examined and rated. If the CSRQ Center’s reviewers 
deemed the rigor of a study’s research design to be 
strong or conclusive using QRT Part 2, then the study 
proceeded to QRT Part 3. Using QRT Part 3, reviewers 
looked across studies on an ESP model and rated the 
cumulative evidence as either very strong, moderately 
strong, moderate, limited, zero, or no rating. Using 
research and evidence that met the CSRQ Center’s 
standards set forth in QRT Parts 1 and 2, these rubrics 
helped evaluate the extent to which a model can 
demonstrate positive impact in the five categories that 
were described previously: 

■ Evidence of positive effects on student achievement 

■ Evidence of positive effects on additional outcomes 

■ Evidence of positive effects on parent, family, and 
community involvement 
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■ Evidence of a link between research and the 
model’s design 

■ Evidence of services and support to schools to 
enable successful implementation 



H ow Does the Rating System Work? 

Our rating process is complex and is based on the 
assumption that to make timely decisions, education 
consumers need a relatively small number of straight- 
forward ratings developed through reliable methods. 
Our system combines two elements to provide a single 
rating for each of the categories and subcategories: 

■ The strength of the evidence based upon the 
causal validity of the research design (e.g., how 
reliable and credible is it). Strength of evidence 
depends on several elements: (a) the rigor of the 
research design and the reliability of the evidence 
produced, (b) the quantity of the research evidence 
provided by a model, and (c) the consistency of the 
evidence pointing to positive outcomes. 



■ The strength of the reported impact or effect 
(e.g., does the model raise student achievement 
a little or a lot). To measure the impact of the 
model, we calculated effect sizes — a measure of 
standardized differences between groups that 
allows researchers to compare impact on different 
outcomes (e.g., reading achievement on different 
tests). We then established a range of effect sizes 
that would be used to categorize the strength of 
impact and contribute to the overall rating, (for 
information on effect sizes, see “About Effect Sizes.”) 

“Methodology” describes the rating process in detail. 

The CSRQ Center applied separate rubrics for each 
category to arrive at its ratings. Ratings are expressed 
by a common set of symbols. In general, the rubrics 
we used resulted in the following ratings: 

■ Very Strong This is the highest rating provided 
by the CSRQ Center. It means that the model demon- 
strates very strong (highly credible) evidence of a 
very strong (large) impact in a reviewed category. 

■ Moderately Strong This is the second highest 
rating. It indicates that the combination of strength 



About Effect Sizes 

Effect sizes (ESs) are a way to standardize measures to show gains and losses on achievement or other outcomes, where 
differences between experimental and control groups are expressed as standard deviation (SD). For example, an ES of 
1.00 Indicates that students using an ESP model scored one full SD higher than comparison students not using that model. 
This is equivalent to an estimated increase of 100 points on the SAT, 21 NCEs (normal curve equivalent ranks), 15 points 
of IQ, or enough to move a student from the 20th percentile to above the 50th percentile (Slavin & Fashola, 1998). 

ESs appear throughout this report to serve two purposes. First, we report ESs when describing results within individual 
studies. The range of outcomes in these studies varies greatly. Second, and most Importantly, we report average ESs 
that Indicate the effects of an ESP model across studies on various outcomes. ESs are used by the CSRQ Center as one 
component to rate models on their evidence of effectiveness. Based on a review of existing literature on ESs for ESP 
models and in consultation with experts, we set ranges for very strong (+0.25 and above), moderately strong (+0.20 to 
+0.24), and moderate (+0.15 to +0.19) as components of our model rating rubrics. Because of differences among study 
designs and assessments, our determination of ESs for each model can be considered only a rough estimate of Impact, 
allowing for comparison among the various models. 
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of evidence and strength of impact is moderately 
strong, because for either or both, the evidence 
base is not sufficiently rigorous or the overall 
impact is not as large as for very strong models. 

■ Moderate 3 - This rating results when either or 
both the strength of evidence or the strength of the 
impact do not meet the higher standards described 
above. Models receiving this rating may still have 
notable evidence because of rigor or impact. 

■ Limited (^ . This rating indicates that although the 
CSRQ Center found some evidence of effectiveness, 
more rigorous research needs to be conducted on 
the model to fully support its effectiveness on the 
category reviewed. 

■ Zero 0. This rating means that none of the studies 
were of sufficient quality to be counted as reliable 
evidence. 

■ Negative 0. This rating indicates that the CSRQ 
Center found strong evidence of detrimental 
effects in a given category or subcategory. In prac- 
tice, we did not find any evidence of this kind for 
any model. 

■ No Rating This rating indicates that the model 
has no studies (i.e., no evidence) available for review 
in a category or subcategory. 

Table 2 illustrates how a set of fictitious ESP models 
(A-F) might have been rated based on their evidence of 
effectiveness (impact) and the strength of their evidence. 
As noted above and detailed in “Methodology,” models 
vary in cumulative effect sizes. The higher the positive 
effect size, the greater the estimated positive impact on 
the category under analysis. (Whenever possible, 
effect sizes were calculated for Categories 1, 2, and 3.) 
Strength of evidence, as noted previously, is a com- 
pound of several elements. Because a model can vary in 
the quantity of these two components, several models 
may receive the same rating for different reasons. 



Several conclusions can be drawn from Table 2: 

■ Model A and Model B are rated limited. In Model 
As case, we would have found that we had fairly 
high confidence based on its research evidence that 
the model has limited impacts. Although Model B 
seemed to have moderate impact, we had little 
confidence that this was indeed the case given the 
research that suggested this effect (e.g., research 
designs with relatively lower rigor were used). 

■ Models C and D would have received a moderate 
rating but for different reasons. Model C has 
moderately strong evidence but a limited impact; 
Model D has a stronger effect but weaker evidence 
(e.g., only a few studies). 

■ Models E and F have strong effect size results 
(impact), but Model F has stronger evidence (e.g., 
a larger number of highly rigorous studies were 
conducted, leading to greater confidence) supporting 
a rating of very strong versus moderately strong 
(for Model E). 

In practice, the seven ESP models we reviewed might 
have been arrayed in a similar fashion because they 
demonstrated a large range in effect sizes and in the 
level of confidence we could place on their research 
findings. 

Similarly, the rating system for Categories 4 and 5 was 
complex and depended on several elements: (a) evidence 
of explicit links between research and the model’s 
design, (b) evidence that the model provider offers 
services and supports to schools to enable successful 
implementation, and (c) evidence that the model 
provider offers professional development and technical 
assistance to enable successful implementation. 

To determine evidence of services and supports, the 
following areas were examined: (a) provider tracks 
and supports full implementation in all schools and 
(b) provider helps schools to allocate resources needed 
to fully implement the model. For evidence of 
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Table 2. The CSRQ Center’s Rating System for Categories 1-3 



Strenath of Evidence 

^Low Confidence in Evidence High^ 


Impact 

^Weak Strength of Effect Very Strong^ 


Limited 


Moderate 


Moderateiy 

Strong 


Very Strong 


Very Strong 


Limited 

Model A 


Moderate 


Moderateiy 

Strong 


Moderateiy 

Strong 


Very Strong 

Model F 


Limited 


Moderate 

Model C 


Moderateiy 

Strong 


Moderateiy 

Strong 


Moderateiy 

Strong 

Model E 


Limited 


Moderate 


Moderate 


Moderate 

Model D 


Moderate 


Limited 


Limited 


Limited 

Model B 


Limited 


Limited 



professional development and technical assistance, 
the following areas were examined: (a) provider offers 
extensive training opportunities and supporting mate- 
rials to support its core components and (b) provider 
supports schools in the development of its internal 
capacity to provide professional development. 

The same rating scale and symbols were used to rate 
Categories 4 and 5 as were used to rate Categories 
1-3. But the meanings of the ratings are different so 
that they match the category: 

■ Very Strong This is the highest rating provided 
by the CSRQ Center. It means that the model pro- 
vided evidence of explicit links between research 



and the model’s design, comprehensive services 
and supports to schools to enable successful imple- 
mentation, and/or comprehensive professional 
development and technical assistance to enable 
successful implementation for 100% of the model’s 
core components. 

■ Moderately Strong This is the second highest 
rating. It indicates evidence of explicit links between 
research and the model’s design, comprehensive 
services and supports to schools to enable successful 
implementation, and/or comprehensive professional 
development and technical assistance to enable 
successful implementation for 75% of the model’s 
core components. 
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■ Moderate 3 . This rating indicates evidence of 
explicit links between research and the model’s 
design, comprehensive services and supports to 
schools to enable successful implementation, and/or 
comprehensive professional development and tech- 
nical assistance to enable successful implementa- 
tion for 50% and at least two of the model’s core 
components. 

■ Limited (J . This rating indicates evidence of 
explicit links between research and the model’s 
design, comprehensive services and supports to 
schools to enable successful implementation, or 
comprehensive professional development and tech- 
nical assistance to enable successful implementa- 
tion for less than half (below 50%) and at least 
one of the model’s core components. 

■ Zero 0. This rating means that we found a non- 
specific research base, no evidence of services and 
supports, and/or evidence that does not meet the 
CSRQ Center’s standards of rigor and quality. 

■ No Rating ®. This rating indicates that the CSRQ 
Center was unable to conduct a conversation with the 
ESP or to obtain complete information to verify evi- 
dence. Thus, no rating would be given to the model. 




hat Are the CSRQ Center’s Findings? 



Our rating process for Categories 1, 2, and 3 is complex 
and combines two elements to provide a single rating: 



■ The strength of the evidence based on the causal 
validity of the research design (e.g., how reliable 
and credible is it) 



■ The strength of the reported impact or effect 
(e.g., does the model raise student achievement a 
little or a lot) 



The CSRQ Center identified few rigorous studies that 
were relevant for rating each ESP’s overall evidence of 



positive effects on student achievement. In Category 1, 
after screening approximately 900 studies and docu- 
ments for quality, only 9 studies met the CSRQ Center’s 
standards. (Appendix I, Table 1 summarizes the quan- 
titative study findings that were used to rate evidence 
of overall positive effects on student achievement.) 
These 9 studies focused on one ESP model (Edison). 

Eor Category 1 (Evidence of Positive Effects on Student 
Achievement), we rated 

■ Qne model as moderate (Edison), 

■ Eour models as zero (Leona, Mosaica, NHA, and 
White Hat), and 

■ Two models as no rating (Imagine and SABIS). 

In reviewing our findings for Category 1, readers 
should keep in mind that many of the models in the 
report serve high-poverty students in low-performing 
schools. Thus, the evidence of effectiveness that the 
studies present is for success in educating students in 
highly challenging conditions. 

The research base on which to rate models in 
Categories 2 (Evidence of Positive Effects on Additional 
Qutcomes) and 3 (Evidence of Positive Effects on 
Parent, Eamily, and Community Involvement) is sparse. 
There were no studies that met the CSRQ Center’s 
standards in these categories. 

Category 4 rated evidence of link between research and 
the model’s design. The rating system for Category 5 
(Evidence of Services and Support to Schools to Enable 
Successful Implementation) depended on two subcate- 
gories: (a) evidence of readiness for successful imple- 
mentation and (b) evidence that the model provider 
offers professional development and technical assistance 
to enable successful implementation. The same rating 
scale and symbols were used to rate Categories 4 and 5 
as were used to rate Categories 1-3; however, the mean- 
ings of the ratings are category specific. The CSRQ 
Center contacted each ESP to conduct a conversation 
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with the provider to verify information to complete 
ratings in Categories 4 and 5. However, four models 
(Imagine, Mosaica, SABIS, and White Hat) did not 
engage in conversations with the CSRQ Center, and 
three of those models (Imagine, Mosaica, and White 
Hat) did not provide any information. Thus, given the 
limited amount of publicly available information in 
these categories, we assigned no rating to these ESPs 
for Categories 4 and 5. 

For Category 4, we rated 

■ One model as very strong (Edison), 

■ One model as limited (Leona), and 

■ One model as zero (NHA). 

For Category 5a (evidence of readiness for successful 
implementation), one model (Edison) was rated very 
strong and two models (Leona and NHA) were rated 
moderate. For Category 5b (evidence that the ESP offers 
professional development and technical assistance to 
enable successful implementation), one model (Edison) 
was rated very strong and two models (Leona and 
NHA) were rated moderately strong. 

Given the importance of implementation to the success 
of any whole school reform, consumers who select 
models that have low rankings in evidence of effects on 
student outcomes may still experience success if they 
implement the models faithfully. Appendix I, Table 2 
summarizes basic model information and model ratings 
for Categories 1-5. 



W hat Are the Limitations of This Report? 

Although this report builds on the strong prior work of 
others (e.g., Borman et ah, 2002; Herman et ah, 1999) 
and the best thinking of the education research com- 
munity regarding how to conduct consumer- friendly 
evidence reviews, it falls short of the ideal in a number 
of areas. We hope that over time — with the feedback 



of education consumers, researchers, and model 
providers — we will be able to issue future reports that 
are increasingly accurate and useful. 

Relying on existing evidence in providing ratings was 
a major limitation of this report. Our descriptive 
information was based on a review of publicly available 
information that is often provided by the models 
themselves. This report on ESPs was particularly 
challenging because only some providers engaged with 
the CSRQ Center in the review process. Although aU 
ESPs were given the opportunity to engage in a con- 
versation with the CSRQ Center to discuss their model 
and to provide additional materials and research to be 
considered for this report, only a few chose to do so. 
Thus, in most instances, the CSRQ Center relied 
entirely on publicly available information, making it 
difficult to assign some ratings. 

Given limited resources, verifying the claims made by 
aU ESPs was impossible. We did attempt to gather 
independent information through conversations with 
a small group of randomly selected districts or charter 
authorizers served by the models reviewed. However, 
these were informal conversations that were conducted 
with only a very small number of individuals. Given 
our limitations, other participants and stakeholders 
involved in whole school reform — such as teachers, 
students, parents, and school board members — could 
not be reached. We encourage consumers to probe more 
deeply during the ESP adoption process for further 
information to support their final choice of a model. For 
example, schools and districts can and should request 
detailed cost, operational, and evaluation information 
from an ESP as part of a contracting process. 

Likewise, our quantitative information was limited to 
a review of available prior research that had been con- 
ducted on the seven models. Although we searched 
extensively to uncover all sources of existing evidence, 
we were not able to conduct original research or to apply 
common evaluation measures across all models to ease 
comparability. Also, because models are evolving and 
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refining their designs, we cannot be certain whether 
the high or low ratings given to a model are truly rep- 
resentative of the current version of that model. Many 
models may be “new and improved” but may not yet 
have rigorous evidence to demonstrate such a claim. 

As Professor Larry Hedges notes, 

Evidence -based social policy formation 
requires a base of evidence that key actors . . . 
view as sufficiently valid to warrant its 
active application in policy formation. The 
evidence must at least meet minimum 
standards of internal validity (freedom from 
bias) and external validity (generalizability 
to other settings than the one studied). It is 
not always easy to specify exactly what evi- 
dence meets these standards. (2000, p. 193) 

The CSRQ Center undertook this review with the full 
knowledge of an ongoing scientific debate on such 
questions as how to appropriately weigh evidence from 
different types of research designs, add up research 
findings, and report results. We confronted a number 
of these questions in our review, and each time con- 
sulted our expert technical advisors to arrive at a 
workable answer that allowed us to reach our goal: 
a consumer-friendly report that is based on the best 
available evidence and scientific thinking. However, 
to do so, we had to resolve such issues as (a) how to 
present a composite measure that included rigor of 
research design with strength of impact and (b) how 
to set cut points to determine how large of an effect 
size was needed to gain a rating of very strong, mod- 
erately strong, or moderate on our rating of overall 
effects. We have made our assumptions and our work as 
transparent as possible so that others can help improve 
our thinking and methods for future reports. 

Finally, we knew that to be usable, this report had to 
strike a balance between brevity and depth. Too little 
information risked falling short of our goal to provide 
consumers with an effective decision-making tool. Too 



much information risked confusing decision makers 
with an overwhelming set of details. In practice, we 
erred on the side of providing less numbers and tech- 
nical information in our analyses, leaving that for 
“Methodology” and the appendixes. However, we also 
erred on the side of providing as detailed a description 
of the models as possible, hoping that consumers will get 
a clear understanding of the distinctive elements of each, 
and thus be able to make the wisest decision possible. 
We hope that we made the right sacrifices to meet the 
evidence needs of end users of this report, while 
upholding the highest standards of scientific research. 
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This report is the first systematic attempt to characterize 
the evidence on the effectiveness and quality of seven 
widely used Education Service Providers (ESPs). 
Although this report is intended for a general reader- 
ship, cutting edge scientific concepts and processes 
have been used to produce the reviews in this report. 
In this section, we detail the research methods used 
to support these reviews. This section highlights some 
of the challenges posed in conducting systematic 
reviews of evidence and gives our technical readers 
the background needed to judge the quality of our 
scientific efforts. 

The Comprehensive School Reform Quality (CSRQ) 
Center’s researchers recognize that student achieve- 
ment is critical to education consumers. However, 
education consumers also rely on thorough descrip- 
tions of whole school improvement models, such as 
ESPs, and they want to know how their schools may 
change if they implement a specific model. Educators 
also seek information about the experiences of other 
schools that implement whole school improvement 
models. To meet this need, the CSRQ Center com- 
bined qualitative and quantitative research techniques 
to report on (a) the impact that ESP models have on 
student achievement and (b) the experiences of 
schools that implement these models. This approach 
aligns with Creswell’s five purposes for the use of 
multimethods (1994, p. 175): 

1. Triangulation, in seeking convergence of results 

2. Complementary, in that overlapping and different 
facets of a phenomenon may emerge 

3. Developmentally, wherein the first method is used 
sequentially to help inform the second method 

4. Initiation, wherein contradictions and fresh 
perspective emerge 



5. Expansion, wherein the mixed methods add scope 
and breadth to a study 

Through the use of multimethods, the CSRQ Center 
reviewed available evidence on ESP models to deter- 
mine their effects on student achievement and to 
expand and fully describe the components of each 
ESP model and the services that they offer to schools. 

As described in the introduction, the CSRQ Center 
developed the Quality Review Tool (QRT), a three-part, 
multimethod tool to collect and analyze qualitative 
and quantitative data to evaluate ESPs for the education 
consumer. 

1. QRT Part 1 is the qualitative data collection phase. 
The purpose of QRT Part 1 is to gather (a) support- 
ing information on each ESP from publicly avail- 
able sources, ESP directors, and three districts or 
charter authorizers and (b) descriptive information 
about the ESP, such as professional development, 
technical assistance, and research-based design. 

2. QRT Part 2 is the quantitative data collection 
phase. The purpose of QRT Part 2 is to conduct a 
systematic review of the literature on the effective- 
ness of an ESP on student achievement, other out- 
comes — such as attendance and graduation rates — 
and family and community involvement. 

3. QRT Part 3 is the data analysis phase, in which 
the qualitative and quantitative data are synthesized 
to generate effectiveness ratings of the ESP. These 
ratings (Very Strong, Moderately Strong, Moderate, 
Limited, Zero, and No Rating) are developed for 
several categories, including evidence of positive 
effects on student achievement, additional outcomes, 
and parent, family, and community outcomes; 
evidence of a link between research and the model’s 
design; and evidence of the model’s ability to provide 
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services and support (e.g., readiness and professional 
development/technical assistance) to schools to 
enable successful implementation. 




ample of Education Service Provider 



Models 



For the purposes of this report, ESP is defined as a 
nonprofit or for-profit organization that contracts with 
new or existing public, charter, or private schools and/or 
school districts to provide comprehensive services, 
including, but not limited to, educational programming 
and administrative services. Educational programming 
included curriculum design, professional development, 
and student assessment tools. Administrative services 
included, but were not limited to, operation manage- 
ment services (student enrollment, school marketing), 
financial management services (payroll assistance, 
budget oversight), facilities management services (main- 
tenance and use of facilities), and human resources 
management (hiring and training staff, staff benefits). 
The comprehensive services provided by ESPs are 
comparable to services provided by whole school 
improvement models. 

The CSRQ Center gathered an initial list of 54 ESPs by 
consulting databases, including Northwest Regional 
Educational Laboratory’s (NWREL’s) Catalog of School 
Reform Models; the Center for Education Reform; and 
reports, including Education Management Organizations: 
Growing a Eor-Profit Education Industry With Choice, 
Competition, and Innovation (Hentschke, Oschman, 

& Snell, 2002), Calculating the Benefits and Costs of Eor- 
Profit Public Education (Molnar, 2001), and Profiles of 
Eor-Profit Education Management Companies, Sixth 
Annual Report, 2003-2004 (Molnar, Wilson, & Allen, 
2004). Erom this list, a final sample was selected by 

1. Exploring the replicability of the ESP, as determined 
by the total number of states implementing 
the ESP; 



2. Determining market share, as defined by the total 
number of schools implementing the ESP; and 

3. Investigating the comprehensiveness of the ESP’s 
design. 

During each step of the information gathering process, 
researchers consulted previous reports, databases, Web 
sites of the ESPs, and four recognized researchers in 
school management organizations. 

Eor step 1, (replicability), the CSRQ Center’s researchers 
consulted the Web sites of the initial 54 ESPs to deter- 
mine whether they operated in three or more states. 
This step narrowed the initial list from 54 to 15 ESPs. 

Eor step 2 (market share), the CSRQ Center’s 
researchers searched the Web sites of the remaining 
15 ESPs for information on the total number of schools 
that used the respective ESPs. The CSRQ Center defined 
the selection criterion for market share as ESP models 
that operated in 20 or more schools. This step narrowed 
the list from 15 to 13 ESPs. 

Eor step 3 (comprehensiveness), the CSRQ Center’s 
researchers examined whether the ESP’s design fea- 
tures met the following components identified by the 
U.S. Department of Education: governance, adminis- 
trative services, technical assistance, classroom prac- 
tices, professional development, leadership develop- 
ment, benchmarks/ assessments, and curriculum 
(U.S. Department of Education, n.d.). Eor coding 
purposes, components were defined as follows: 

■ Governance was defined as operations and man- 
agement conducted in schools. Key words associated 
with governance were operations, structure, man- 
agement, scheduling, committees, blocks, and 
administration. 

■ Administrative services (AS) was defined as 
central office tasks, such as budgets, payroll, and 
student recruitment. AS included, but were not 
limited to, operation management services (student 
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enrollment, school marketing), financial manage- 
ment services (payroll assistance, budget oversight), 
facilities management services (maintenance and 
use of facilities), and human resources management 
(hiring and training staff, staff benefits). Keywords 
associated with AS were payroll, budget, personnel 
management, recruitment, facility management, 
maintenance, and transportation management. 
Researchers should note that this component only 
applies to the CSRQ Center’s report on ESPs. 

■ Technical assistance (TA) was defined as classroom 
operational or management assistance through 
mentoring, coaching, or other services provided 

to teachers. Key words associated with TA were 
troubleshooting, coaching, and mentoring. 

■ Classroom practices (CP) was defined as peda- 
gogical, structural, and behavioral management 
practices that a teacher enacts in a classroom. Key 
words associated with CP were pedagogy, class- 
room management, classroom structure, teaching 
strategies, and philosophy of instruction. 

■ Professional development (PD) was defined as 
teacher training on a specific topic. This training 
typically occurs in a workshop or conference 
environment. Key words associated with PD were 
training (on specific topics), conferences, and 
workshops. 

■ Leadership development (LD) was defined as 
administrative training or development for school 
personnel in leadership positions (principals, 
grade-level chairs, and lead teachers). Keywords 
associated with LD were leadership training and/or 
development. 

■ Benchmarks/assessments was defined as tests and 
evaluations used to measure students’ skills and 
understanding and academic progress. Key words 
associated with benchmarks/assessment were 



measurable goals, formative evaluation, and bench- 
marks of progress. 

■ Curriculum was defined as the scope and sequence 
of learning objectives and indicators, as well as 
materials provided for lessons to instruct such 
objectives. Key words associated with curriculum 
were materials, scope and sequence, standards, and 
learning objectives. 

Each ESP was given a point for each component or 
criterion that the model met based on information 
found on the model’s Web site and additional 
resources, including, but not limited to. Education 
Management Organizations: Growing a For-Profit 
Education Industry With Choice, Competition, and 
Innovation (Hentschke, et al., 2002); Calculating the 
Benefits and Costs of For-Profit Public Education 
(Molnar, 2001); Profiles of For-Profit Education 
Management Companies, Sixth Annual Report, 
2003-2004 (Molnar et al., 2004); and Web sites of the 
US. Department of Education (http://www.ed.gov) 
and NWREL (http://www.nwrel.org). Each ESP that 
had five or more components in its design was included 
in the final sample. This step narrowed the list from 
13 to 7 ESPs, resulting in the final sample for the review. 



RT Part 1: Qualitative Data Collection 

Phase 

QRT Part 1 was the qualitative data collection phase. 

It included guidelines for (a) conversations with model 
directors, school districts, or charter authorizers and 
(b) the collection of artifacts from the ESPs and addi- 
tional information about the model from publicly 
available resources (Bogdan & Biklen, 1998; Creswell, 
1994, 1998). 

QRT Part 1, including the guidelines for phone con- 
versations, conversation questions, and artifact lists, 
was pilot tested with a model provider reviewed in the 
CSRQ Center Report on Elementary School Comprehensive 
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School Reform Models (released November 2005). Based 
on feedback from the pilot conversations, researchers 
at the CSRQ Center modified the qualitative data 
collection process. An experienced and trained 
qualitative researcher at the American Institutes for 
Research (AIR) provided training on information 
gathering techniques, coding artifacts, and synthesizing 
qualitative data to develop a complete description of 
each ESP in the sample. The qualitative researchers 
met weekly to ensure consistency across the qualitative 
data collection efforts. 

For QRT Part 1 (qualitative data collection), qualitative 
researchers performed four main steps: 

1. Complete an initial description of the ESP model 
by using a standardized form. The CSRQ Center 
developed the Model Description Form, a compre- 
hensive survey instrument for compiling existing 
information about an ESP, including mission, 
history, market share, costs to the school, and 
design of each of the ESP model’s components. 

For example, researchers gathered information 
about the ESP’s organization and governance, such 
as how the ESP provides site-based autonomy, 
whether additional personnel are needed, and 
whether the ESP requires changes to the structure 
of the school. For questions about professional 
development, researchers gathered information 
about which school personnel are required to 
attend professional development, what types of 
professional development are offered prior to and 
during implementation, and what strategies are 
available to help a school build capacity to provide 
its own professional development. In all, researchers 
gathered information about the ESP’s organization 
and governance; administrative services; profes- 
sional development; technical assistance; curriculum; 
instruction; inclusion; technology; time and sched- 
uling; instructional grouping; student assessment; 
data-based decision making; and parent, family, 
and community involvement. The researchers also 



requested benchmarks and explicit citations that 
link the model’s design to a research base. The 
researchers completed this form using the ESP’s 
Web site and other publicly available information. 

2. Conduct a phone conversation with the provider 
of the ESP model to verify previously gathered 
information. Conversations were structured around 
the Model Description Form (completed in step 1). 
On average, phone conversations lasted 90 minutes. 

3. Conduct phone conversations with three districts 
or charter authorizers who use or authorize the 

use of the ESP. The conversations verified infor- 
mation gathered in steps 1 and 2. The districts or 
charter authorizers were randomly selected from a 
list provided by the ESP. The conversations were 
guided by the Model Description Form. 

4. Complete a final description of the ESP by using 
a standardized form. The Model Description Form- 
Complete synthesized all sources of qualitative data 
gathered, such as the conversations with the model 
provider, the three districts, or charter authorizers 
and artifacts collected from the ESP. The Model 
Description Form-Complete was checked for quality 
control twice to ensure that each item had 100% 
agreement between the two qualitative researchers. 
This form was then used to organize the data 
through the identification of core components. 

Core components are considered essential to suc- 
cessful implementation of the model according to 
the CSRQ Center’s standards. Additionally, these 
data were coded to answer several questions: 

■ Is there a strong link between research and the 
ESP’s design? 

■ Does the ESP track and support full implemen- 
tation in all schools? 

■ Does the ESP help schools allocate resources to 
implement the model? 



QRT PART 1: QUALITATIVE DATA CQLLECTIQN PHASE 



26 



METHODOLOGY 



■ Does the ESP provide comprehensive training 
opportunities and supporting materials? 

■ Does the ESP develop the schools’ internal 
capacity to provide professional development? 



RT Part 2: Quantitative Data 
Collection Phase 

QRT Part 2 was the quantitative data collection phase. 
Using systematic review methods (Borman, Hewes, 
Overman, & Brown, 2002; Lipsey & Wilson, 2001), 

QRT Part 2 included protocols to conduct systematic 
literature reviews and to code research studies for 
statistical and causal validity information. 

QRT Part 2, including the protocols for literature 
reviews and coding instruments, was pilot tested using 
the same whole school improvement model provider 
as was used for the qualitative data collection efforts 
(QRT Part 1). Based on feedback from the pilot test 
and from the CSRQ Center Report on Elementary 
Comprehensive School Reform Models, the process for 
conducting the literature review was improved and the 
coding instruments were refined. An experienced and 
trained quantitative researcher at AIR conducted 
training on how to use the coding instruments to 
ensure consistency in the data collection. The training 
included a presentation of the definitions of different 
research designs, causal validity issues, and background 
information on effect size calculations. 

Eor QRT Part 2, quantitative researchers completed 
five main steps: 

1. Conduct a thorough literature search. Eor each 
ESP, quantitative researchers searched educational 
databases (e.g., JSTOR, ERIC, EBSCO, Psychinfo, 
Sociofile, NWREL, DAI) and Web-based repositories 

'Some studies reviewed by the CSRQ Center evaluated more than one ESP model. 
^One study for Mosaica schools was not available in full copy. 



(e.g., Google, Yahoo, Google Scholar). Prom these 
sources, quantitative researchers screened for initial 
relevance nearly 940 article abstracts or summaries 
across the 7 models in the final sample. To pass the 
initial screen, the studies had to meet several criteria: 
be published or distributed between 1980 and 
April 2005, examine at least one of the ESPs being 
investigated, use quantitative methods, and be 
reported as a full-text research paper (i.e., not a 
PowerPoint presentation or executive summary). 
Prom these articles, researchers identified 38 articles 
to code.^ Of those, 37 were available and retrievable 
for coding.^ Appendix I provides a summary table 
of the number of articles that passed through each 
phase of the QRT Part 2 process. 

2. Complete a Study Description Outcome Form 
(SDOF), the first standardized coding sheet. The 

CSRQ Center’s quantitative researchers used the 
SDQF to code and document each study’s research 
design, outcome variables, and demographic infor- 
mation. The CSRQ Center assigned a lead and 
secondary coder for each article. The SDQF was 
completed by the lead coder. Then, the secondary 
coder verified all the information for 100% agree- 
ment. At this stage of coding, the primary focus 
was to screen each study for a reliable research 
design. Studies that were not eligible for full review 
were often evaluations of implementation theories 
supporting the ESP with no quantitative data on 
outcomes or used research designs that were not 
sufficiently rigorous (e.g., one group pretest- 
posttest research designs). Research designs that 
passed this stage included experimental and quasi- 
experimental research designs with both pre- and 
posttests that evaluated the ESP with a control 
group (Cook & Campbell, 1979; Shadish, Cook, 

& Campbell, 2002) and longitudinal and cohort 
designs with multiple testing periods. Studies with 
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research designs that passed this screen and 
included student achievement outcomes became 
eligible for full review. A total of 17 articles passed 
this step and were eligible for full coding in step 3. 

3. Complete the Quality Indicators Form (QLIF), 
the second standardized coding sheet. Researchers 
used the QLIF to code studies that appeared to use 
rigorous research designs. The QLIF served two 
purposes: It examined the quality of the research and 
gathered statistical information. Researchers exam- 
ined the quality of the research, such as the internal 
and external validity, face and psychometric validity 
of the outcome measures, and other quality indica- 
tors (Herman et al., 1999). Coders also collected 
statistical information, such as effect sizes reported 
by the authors or raw statistical information. For 
each study that was relevant for full review, two 
quantitative researchers independently coded one 
QLIF for each achievement outcome in that study. 

4. Reconcile the two QLIF coding sheets to attain 
100% agreement on each coded item. If the two 

quantitative researchers could not reach a consensus, 
a review coordinator reviewed the coding sheets to 
facilitate reconciliation. After the reconciliation 
process, a final QLIF reflected the 100% agreement. 

5. Rate each article on an overall causal validity 
score. The final step was to systematically map the 
information from the final QLIF (the reconciled 
version) based on a set of rubrics designed to score 
each study for its causal validity (Shadish et al., 2002) 
as conclusive, suggestive, or inconclusive. Studies 
determined to be suggestive or conclusive met the 
CSRQ Centers standards for rigor of research design. 

Conclusive studies had high levels of rigor, that is, 
experimental and quasi-experimental designs that had 
zero critical threats to validity and fewer than two 
noncritical threats to validity. Effect sizes were reported 



or calculated only from studies that had a conclusive 
causal validity rating (Cooper, 1998; Light & Pillemer, 
1984; Shadish et al., 2002). If the researcher could not 
calculate an effect size because of missing data, then 
the researcher conducted one of the following steps: 

(a) contacted the author for the statistical information 
needed, (b) imputed missing data, particularly standard 
deviations and sample size using protocols established 
in previous meta-analysis (Borman et al., 2002), or 
(c) chose not to include the study in the synthesis if 
options a and b were not feasible. 

Suggestive studies are those that had zero critical threats 
but more than two noncritical threats. Studies without 
control groups, including longitudinal and cohort 
research designs, were capped at suggestive, unless the 
analytic techniques generated higher levels of rigor.^ 

Inconclusive studies had critical threats to validity, such 
as using testing instruments with poor face validity and 
reliability, insufficient program fidelity, nonequivalence 
of treatment/control groups, lack of proper baseline, 
and/or timing of outcome measures (less than 1 school 
year after ESP implementation or less than 1 academic 
year between pretest and posttest). Noncritical threats to 
validity included historical events, disruption/novelty 
effects, instrumentation changes, maturation, selection 
bias, and statistical regression (Shadish et al., 2002). 

RT Part 3: Data Analysis Phase 

QRT Part 3 synthesized the qualitative and quantitative 
data to evaluate each ESP in five main categories. 

1. Evidence of positive effects on student achievement: 

a. Evidence of positive overall effects 

b. Evidence of positive effects for diverse student 
populations 




’For example, backward-looking interrupted time series designs were considered more rigorous than longitudinal or longitudinal cohort studies that examined 
trends over time. 
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c. Evidence of positive effects for specific subject 
areas 

2. Evidence of positive effects on additional outcomes 
(e.g., student discipline, student attendance, school 
climate, retention/promotion rates, and teacher 
satisfaction) 

3. Evidence of positive effects on parent, family, and 
community involvement 

4. Evidence of a link between research and the 
model’s design 

5. Evidence of services and supports to schools to 
enable successful implementation: 

a. Evidence of readiness for successful 
implementation 

b. Evidence of professional development/technical 
assistance for successful implementation 

Category 1 used the quantitative information gathered 
in QRT Part 2. Eor each ESP in the sample, the quan- 
titative information — including the number of studies 
coded, the number of studies that were rated as sug- 
gestive or conclusive, the percentage of findings in the 
suggestive or conclusive studies that demonstrated a 
positive impact, and the average effect size of those 
significant findings — was mapped onto rubrics to 
determine what rating the model should receive — 
either very strong, moderately strong, moderate, limited, 
zero, or no rating — for effects on student achievement. 
Quantitative researchers systematically aggregated 
results according to the QRT 3 rubric for the overall 
effect by grade, subject (reading, writing, math, science, 
and social studies), and diverse student populations 
(e.g., high poverty, minority, learning disabled and 
other special needs, and urban and rural students). 

Category 2 evaluated the positive effects of each ESP 
on additional outcomes, and Category 3 evaluated the 
evidence of positive effects of each ESP on parent. 



family, and community involvement. Similar to 
Category 1, quantitative researchers mapped onto 
rubrics the information about the number of studies 
that evaluated these outcome variables, the number of 
studies that were suggestive or conclusive, the percent- 
age of findings that demonstrated a positive impact, 
and the average effect size of those positive findings. 

In general, the rubrics for the quantitative information 
for Categories 1-3 were as follows: 

■ Very Strong. If a model had at least 10 studies that 
met the CSRQ Center’s standards for rigor of 
research design, with at least 5 rated conclusive 
(and/or conclusive studies constitute at least 50% of 
the total studies coded), and 75% of the outcomes 
showed statistically significant positive model 
effects ip < .05), with an overall mean model 
achievement effect size for positive effects that is 
greater than or equal to +0.25, then the model 
received a very strong rating. A very strong rating 
is symbolized by a fully shaded circle (^). 

■ Moderately Strong. If a model had 5 to 9 studies 
that met the CSRQ Center’s standards for rigor of 
research design, with at least 3 rated conclusive 
(and/or conclusive studies constituted at least 50% 
of the total studies coded), and 51% to 74% of the 
outcomes showed statistically significant positive 
model effects {p < .05), with an overall mean 
model achievement effect size for positive effects 
that is between or equal to +0.20 and +0.24, then 
the model received a moderately strong rating. A 
moderately strong rating is symbolized by a three- 
fourths shaded circle (9). 

■ Moderate. If a model had 2 to 4 studies that met 
the CSRQ Center’s standards for rigor of research 
design, with at least 1 rated conclusive (and/or 
conclusive studies constituted at least 50% of the 
total studies coded), and 26% to 50% of the out- 
comes showed statistically significant positive 
model effects {p < .05), with an overall mean 
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model achievement effect size for positive effects 
that is between or equal to +0.15 and +0.19, then 
the model received a moderate rating. A moderate 
rating is symbolized by a half shaded circle ( 3 )- 

■ Limited. If a model had 1 study that met the CSRQ 
Center’s standards for rigor of research design and 
1% to 25% of the outcomes showed statistically sig- 
nificant positive model effects {p < .05), then the 
model received a limited rating. A limited rating is 
symbolized by a one-fourth shaded circle (Q). 

■ Zero. If a model had zero studies that met the CSRQ 
Center’s standards for rigor of research design or 
0% of the outcomes in the studies that met the 
CSRQ Center’s standards for rigor of research design 
showed statistically significant positive effects, as 
required for a limited rating, then the model 
received a zero rating. A zero rating is symbolized 
by a circle with a diagonal slash (0). 

■ Negative. If a model had at least 10 studies that 
met the CSRQ Center’s standards for rigor of 
research design, with at least 5 rated conclusive 
(and/or conclusive studies constituted at least 50% 
of the total studies coded), and 75% of the outcomes 
showed statistically significant negative model 
effects {p < .05), with an overall mean model 
achievement effect size of less than or equal to 
zero, then the model received a negative rating. 

A negative rating is symbolized by a circle with a 
minus sign ( 0 ). Studies that receive a negative 
rating suggest that the model has detrimental 
effects. In practice, this review did not find any 
evidence of this kind for any model. 

■ No Rating. If a model had no studies (i.e., no 
evidence was available), then the model received 
a no rating. A no rating is symbolized by a circle 
with “NR” (®). 

Category 4 evaluates the link between research and 

the ESP’s design. This category used the qualitative 



information from QRT Part 1. Qualitative researchers 
applied the information synthesized in the Model 
Description Form (from QRT Part 1) onto the following 
rubric. 

■ Very Strong. If a model provided documentation 
that explicitly described and convincingly supported 
links between the research base and all (100%) 
core components of its design, then the model 
received a very strong rating. A very strong rating 
is symbolized by a fully shaded circle (9). 

■ Moderately Strong. If a model provided documen- 
tation that explicitly described and supported links 
between the research base and most (75%) of the 
core components of its design, then the model 
received a moderately strong rating. A moderately 
strong rating is symbolized by a three-fourths 
shaded circle (3)- 

■ Moderate. If a model provided documentation 
that explicitly described and supported links 
between the research base and half (50%) of the 
core components of its design, then the model 
received a moderate rating. A moderate rating is 
symbolized by a half shaded circle ( 3 )- 

■ Limited. If a model provided documentation that 
explicitly described and supported links between 
the research base and less than half (below 50%) of 
the core components of its design, then it received 
a limited rating. A limited rating is symbolized by 
a one-fourth shaded circle (Q). 

■ Zero. If a model provided documentation that 
referred to a nonspecific research base to support the 
inclusion of the core components in its design, then 
the model received a zero rating. A zero rating is 
symbolized by a circle with a diagonal slash (0). 

■ No Rating. If the CSRQ Center was unable to 
conduct a conversation with the model provider or 
obtain complete information to verify evidence. 
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then the model received a no rating. A no rating is 
symbolized by a circle with “NR” (®). 

Two main questions guided the ratings for Category 5 
(evidence that the model provider offers services and 
support to schools to ensure successful implementation). 
The first question — does the ESP provide evidence of 
readiness for successful implementation — included the 
following subcategories: 

■ Provider tracks and supports full implementation 
in schools. 

■ Provider helps schools allocate resources that are 
needed to fully implement the ESP. 

Unlike many whole school reform models that often 
require a consensus among an existing school’s staff to 
adopt the model, ESPs do not need to ensure an initial 
commitment from schools because these models often 
open new schools. ESPs offer an alternative route to 
gain consensus; they use an induction process to 
familiarize the new school staff with the model. Thus, 
for this report, under Category 5, the following sub- 
category did not apply as it was used for other reports 
from the CSRQ Center: provider ensures initial 
commitment from schools. 

Qualitative researchers used the information synthe- 
sized in the Model Description Porm (from QRT Part 1) 
to rate the two subcategories using a specific rubric. 
These two ratings were then averaged to determine the 
rating for evidence of readiness for successful imple- 
mentation. In general, a model’s rating was based on 
evidence of the following: formal or informal bench- 
marks for all or some of its core components and a 
formal or informal process for the allocation of such 
school resources as materials, staffing, and time. 

The second question — does the ESP provide schools 
with professional development and technical assistance 
needed to help teachers implement the model — 
included the following subcategories: 



■ Provider offers comprehensive training opportunities 
and supporting materials. 

■ Provider ensures that professional development 
effectively supports full model implementation. 

■ Provider develops a school’s internal capacity to 
provide professional development. 

Again, each subcategory received a rating. The three 
ratings were averaged to determine the rating for 
evidence of professional development and technical 
assistance for successful implementation. In general, 
a model’s rating was based on evidence of the follow- 
ing: a variety of training opportunities, supporting 
materials for professional development in all or some 
of its core components, and a formal or informal plan 
to help build a school’s capacity to provide professional 
development. 

In addition to the ratings across these five categories, 
the qualitative data gathered in QRT Part 1, such as the 
artifacts and phone conversations, were synthesized 
into a narrative description of each ESP. Each narrative 
included indepth information about the ESP’s costs and 
descriptions of the following components: organization 
and governance; administrative services; curriculum 
and instruction; scheduling and grouping; technology; 
monitoring of student progress; parent, family, and 
community involvement; professional development 
and technical assistance; and implementation expecta- 
tions and benchmarks. 

In all, qualitative and quantitative data were mapped 
to rate each ESP on 

■ Evidence of positive effects on student achievement; 

■ Evidence of positive effects on additional outcomes; 

■ Evidence of positive effects on parent, family, and 
community outcomes; 

■ Evidence of link between research and the model’s 
design; and 
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■ Evidence of services and support to schools to 
enable successful implementation. 

The quantitative analysis provided a systematic literature 
review of the reported effects of student achievement 
and other outcome variables. ESPs that could show 
relatively more literature consisting of evaluation 
studies were more likely to achieve higher ratings in 
Categories 1-3 (as long as results demonstrated posi- 
tive impact). Through the qualitative analysis, newer 
ESPs and those that did not have a substantial number 
of evaluation reports could be evaluated on such 
dimensions as quality of professional development 
offered by the ESP. Although previous research on 
student achievement offers important considerations, 
education consumers may also consider whether the 
ESP’s design is based on solid research and provides 
a strong commitment to support schools through 
professional development and technical assistance. 
Providers of newer models may not have had sufficient 
time to conduct research on the effectiveness of their 
models, but they should be able to clearly demonstrate 
that their models can work, that is, that the model’s 
design is based on solid evidence of what works. Hence, 
by using both qualitative and quantitative methods, 
the CSRQ Center strives to provide the education con- 
sumer with a thorough and systematic description of 
the effectiveness of each ESP reviewed in this report. 

By using qualitative and quantitative methods to eval- 
uate the effectiveness of widely implemented ESPs, 
this study also strives to provide usable information to 
education consumers. In 2005, U.S. Education Secretary 
Margaret Spellings stated that the No Child Left Behind 
Act “rests on the common sense principles of account- 
ability for results, data-based decision making, high 
expectations for all, and empowering change” (U.S. 
Department of Education, 2005). 

Meeting these goals will require a significant expansion 
of information for education consumers about what 
works. This report is intended to act as a decision- 
support tool for educators who wish to find effective 



whole school improvement approaches for meeting 
locally defined needs. This report helps to provide 
descriptive and evidence-based information on selected 
ESPs that may help educators make decisions — marking 
a significant change in the culture of the education 
system to meet the needs of educators, policymakers, 
community leaders, families, and most importantly, 
America’s children. 
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Overview: 


Basic Model Information and Review Results 


Modei Name: 


Edison Whole School Management and Edison Alliance 


Modei Mission/Focus: 


According to Edison Schools, its mission is to establish schools where all students 
can receive a high-quality education. Edison seeks to have all of its students achieve 
high academic standards in a full range of academic subjects, particularly in literacy 
and math. Edison also stresses character development. 


Year introduced in Schoois: 


1995 


Grade Leveis Served: 


K-12 



Number of Schools 



Total: 

157 


Urban: 

N/A 


Suburban: 

N/A 


Rural: 

N/A 




Costs 














Total Operating 












Costs: 


Training: 


Materials: 


Personnel: 


Other: 


Yearl 


Varies 


Varies 


Varies 


Varies 


Varies 


Year 2 


Varies 


Varies 


Varies 


Varies 


Varies 


Year 3 


Varies 


Varies 


Varies 


Varies 


Varies 


Years 4+ 


Varies 


Varies 


Varies 


Varies 


Varies 



1 . Evidence of Positive Effects on Student Achievement: 

a. Overall effects 

b. Evidence of positive effects for diverse student populations 

c. Evidence of positive effects In subject areas: 

Reading 

Writing 

Math 

2. Evidence of Positive Effects on Additional Outcomes 

3. Evidence of Positive Effects on Parent, Family, and Community Involvement 

4. Evidence of Link Between Research and the Model’s Design 

5. Evidence of Services and Support to Schools to Enable Successful Implementation: 

a. Evidence of readiness for successful implementation 

b. Evidence of professional development/technical assistance for successful implementation 



3 

0 ^ 

3 

0 

3 



I = Very Strong 3 = Moderately Strong 3 = Moderate (J = Limited 0 = Zero 0 = Negative @ = No Rating 



This description is based on publicly available information, including the model’s Web site, regarding the model and its 
costs. The Comprehensive School Reform Quality Center attempted to obtain specific information, but this was not always 
possible. Areas in which exact information was not provided are marked by “N/A.” 



'Although the rating in this subcategory is zero, readers should note that most of the studies on Edison that met standards and also demon- 
strated evidence of positive overall effects on student achievement, examined the effects of this model on schools that served primarily low- 
income and minority populations. 
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M odel Description 

In 1992, Edison Schools was conceived and founded 
by Chris Whittle, an entrepreneur who launched 
Channel One — the first electronic system that provided 
American middle and high school students with 
domestic and international news each morning. Edison 
is a for-profit education service provider (ESP) that 
was created to partner with school districts and charter 
schools to raise student achievement through a 
research-based school design, management services, 
and professional development. 

Between 1992 and 1995, a team of 30 full-time Edison 
employees — led by Benno C. Schmidt, Jr., former 
president of Yale University — conducted primary and 
secondary research to develop the Edison school design 
and support services. Over the course of 3 years, this 
team — including education researchers, teachers, 
school administrators, technology specialists, and 
experts in school finance and management — visited 
schools throughout the world to interview teachers 
and administrators to identify recurring best practices. 
Simultaneously, Edison staff reviewed the research of 
social scientists in the areas of education reform and 
school management, including work by James P. Comer, 
Robert Slavin, James Coleman, Marshall Smith, and 
Stuart Purkey. In August 1995, Edison partnered with 
four schools to implement a school design based on 
these research and development efforts. 

During the past 10 years, Edison has created two dis- 
tinct school reform initiatives with this school design 
as their basis: Edison Whole School Management 
(including district partnerships and Edison charter 
schools) and Edison Alliance. Through the Whole 
School Management model, Edison partners with 
school districts or charter boards to provide well- 
defined academic, operational, and management 
services to individual schools and in rare cases, entire 
districts. Through the Alliance model, Edison partners 



with school districts to provide customized solutions 
to individual schools that are struggling to meet 
Adequate Yearly Progress. Unlike schools implementing 
the Whole School Management model, schools that 
use the Alliance model remain under the management 
and operational control of the school district. The 
CSRQ Center reviewed both models. 

According to the CSRQ Center’s standards, the follow- 
ing were identified as core components of the Whole 
School Management model: organization and gover- 
nance; professional development; technical assistance; 
curriculum; instruction; inclusion; technology; time 
and scheduling; instructional grouping; student 
assessment; data-based decision making; and parent, 
family, and community involvement. The following 
were identified as core components of the Alliance 
model: professional development; technical assistance; 
inclusion; student assessment; data-based decision 
making; and parent, family, and community involve- 
ment. Core components are considered essential to 
the successful implementation of the model. 

Model Mission/Focus 

According to Edison, its mission is to establish schools 
where all students can receive a high-quality education. 
The model seeks to have all of its students achieve 
high academic standards in a full range of academic 
subjects, particularly in literacy and math. Edison also 
stresses character development. 

To achieve this mission, Edison is committed to imple- 
menting its research-based school design in individual 
schools and districts throughout the United States and 
United Kingdom. This school design, which is the 
basis for both the Whole School Management and 
Alliance models, includes 10 fundamental elements: 

■ Schools organized for every student. Edison seeks 
to meet the needs of all students by creating smaller 
schools within schools, also known as academies. 
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■ A better use of time. Edison seeks to increase time 
for core academic subjects through extended 
school days and dedicated instructional blocks. 

■ A rich and challenging curriculum. Edison 
designed a curriculum for all core content areas 
for grades K-12. 

■ Teaching methods that motivate. Edison provides 
professional development on instructional strategies 
that are designed to meet the needs of aU learners, 
including English language learners and students 
with disabilities. 

■ Assessments that provide accountability. Edison 
designed benchmark assessments and data tracking 
systems that allow teachers to differentiate instruc- 
tion for each student and to prepare students for 
standardized tests. 

■ A professional environment for teachers. Edison 
provides teachers with a path for career development 
through professional development and mentoring. 

■ Technology for an information age. Edison inte- 
grates technology into a school’s academic program 
but does not replace instruction with technology. 

■ A partnership with families. Edison seeks to involve 
family members in their child’s education through 
volunteerism, committees, and quarterly meetings. 

■ Schools tailored to the community. Edison seeks 
to link community service providers with the needs 
of the school. 

■ The advantages of system and scale. Edison 
believes that its schools benefit from being part of a 
national Edison network that can share resources. 

Goals/Rationale 

The Edison school design is based on seven principles 

derived from primary and secondary research 



conducted between 1992 and 1995 by a team of Edison 

staff: 

1. School staff should have a clear and ambitious sense 
of purpose. 

2. Strong leadership is essential to a school’s success. 

3. Teachers should have clear expectations and ongoing 
training to meet these expectations. 

4. All staff members are important to the school’s 
success. 

5. School leaders should encourage teamwork. 

6. Principles and practices of accountability should be 
adopted. 

7. Schools should function as communities in which 
teachers know students as individuals. 



CZ/ osts 

The models’ costs vary by school or district. According 
to Edison, for schools or districts that adopt the Whole 
School Management model, the model costs are com- 
parable to the cost per pupil spent by other schools in 
the same district. The average cost for schools that 
adopt the Alliance model is $800-$1000 per pupil. Eor 
more information on the costs of training, materials, 
and personnel, sites should directly contact Edison. 



E vidence of Positive Effects on Student 
Achievement 

Evidence of Positive Overall Effects 
Rating: O 

It is important to note that a rating of limited or higher 
in this category indicates that the research on the 
whole school improvement model provides evidence 
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of positive impact on student achievement. But, few 
ESP models reviewed for this report had evidence that 
met the CSRQ Center’s standards in this category. 
Edison is commended for offering a number of empir- 
ical studies that met the CSRQ Center’s standards. 

The CSRQ Center reviewed 28 studies of Edison for 
effects on student achievement.^ Nine of these studies 
met the CSRQ Center’s standards for rigor of research 
design. Upon review, the CSRQ Center considered the 
findings of seven of these studies to be conclusive, 
meaning that the CSRQ Center has confidence in the 
results reported. The findings of the other two studies 
are considered to be suggestive, meaning that the 
CSRQ Center has limited confidence in them. 

Qverall, the nine studies reported a mix of results 
showing positive, negative, and no effects of Edison; 
about 40% of the 19 separate achievement test findings 
with a reported level of statistical significance demon- 
strated a statistically significant positive impact.^ The 
average effect size of these positive effects was +0.38. 
These results are consistent with an overall rating of 
moderate for the overall effects of Edison on student 
achievement. The nine studies that met the CSRQ 
Center’s standards are described below. (Appendix A 
reports on the 19 other studies that were reviewed but 
did not meet the CSRQ Center’s standards.) 

The seven studies that met standards and were consid- 
ered to be conclusive used a quasi-experimental design. 
Two studies considered to be suggestive used a longi- 
tudinal design without a comparison group. Most of 
the studies that met standards and were considered to 
be conclusive or suggestive examined the effects of 
Edison on students from low-income families in 



schools with large minority populations; most of these 
studies focused on achievement in reading and math. 
One study focused on an Edison high school that served 
predominantly white students from middle- to high- 
income families, and one study focused on the average 
performance of all Edison conversion schools across 
the United States. The majority of the studies that met 
the CSRQ Center’s standards were technical reports. 

One of the seven conclusive studies examined the 
average performance of all Edison conversion schools 
versus comparison schools in each of their first 5 years 
of operation, bindings indicated no statistically signifi- 
cant differences in state standardized tests in reading 
and math in the 2nd year of operation.^ 

Another study examined reading, writing, and math 
achievement of students in seven Edison schools in a 
large Texas district versus comparison schools. Student 
achievement was measured using results on the Stanford 
Achievement Test, 9th Edition and the Texas Assessment 
of Academic Skills (TAAS). The study found that stu- 
dents in Edison schools did not meet or exceed the 
performance of students in the comparison schools. 

A third study examined reading and math achievement 
data from high school students who took the Delaware 
Student Testing Program. This study found significant 
differences in favor of students in Edison schools in both 
subject areas; however, no effect sizes were reported. 

A fourth study was a longitudinal analysis of achieve- 
ment outcomes in reading and math on the Stanford 
Achievement Test, 8th Edition. Two subsamples of 
students were followed: Subsample A progressed from 
grade 2 to 4, and subsample B progressed from grade 
3 to 5. Students in Edison and comparison groups 



^The CSRQ Center reviewed empirical evidence on both the Edison Whole School Management model and the Edison Alliance model. Because many reports 
did not specify the type of model that was being implemented in the schools examined, the CSRQ Center could not separate findings by type of model. 

^The findings that contributed to the total count for Edison come from studies that evaluated the effects of Edison in single schools as well as studies that 
evaluated the effects of Edison in multiple schools. 

^This study also found a negative effect on math in the 3rd year of operation and no statistically significant differences between Edison and comparison 
schools in the 4th and 5th years. The CSRQ Center did not include findings from these years in the review because of large rates of attrition in schools in 
the study’s sample. 
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showed no statistically significant differences in reading 
achievement. In math, students in the Edison group of 
suhsample B showed significantly greater growth rates. 

Three other studies considered to he conclusive exam- 
ined reading achievement of students in grades K-3 in 
Edison schools versus comparison schools. Reading 
achievement was measured using the DurreU Oral 
Reading suhtest and the Woodcock Reading Mastery 
Test. These studies found a mix of positive, null, and 
negative results for students in grades K-2, and no 
significant differences for students in grade 3. 

Two studies considered to he suggestive examined 
trends in student achievement in reading and math in 
multiple Edison schools. These studies found a mix of 
positive, null, and negative trends that varied hy school. 

Evidence of Positive Effects for Diverse Student 
Populations 

Rating: 0 

It is important to note that a rating in this category 
indicates that the model provided detailed additional 
evidence for specific diverse student populations. None 
of the other ESPs reviewed for this report had evidence 
that met the CSRQ Center’s standards in this category. 
Edison is commended for offering detailed additional 
evidence that met the CSRQ Center’s standards. Also, 
the CSRQ Center urges readers to not necessarily 
judge a no rating or a low rating in this category as 
evidence that the model cannot he effective in Title I 
schools or other schools with similar student popula- 
tions. In fact, many ESPs, including Edison, provide 
most of their services to high-poverty, high-minority 
school populations. Thus, readers may interpret our 
overall rating in the category of positive overall effects 
on student achievement as an indicator of the models’ 
effectiveness in working in challenging settings, such 
as Title I schools. 



A study that met the CSRQ Center’s standards exam- 
ined the impact of Edison on low achieving students.^ 
This study found no statistically significant differences 
between students in the Edison and comparison 
schools. Thus, the rating in this suhcategory is zero. 

Evidence of Positive Effects in Subject Areas: Reading 
Rating: (J 

The impact of Edison on reading achievement was 
mixed. All studies that met the CSRQ Center’s stan- 
dards examined reading achievement. Eour of the 
studies demonstrated some positive impact on reading 
achievement. The average effect size of the positive 
results was +0.40. The difference between reading by 
students in Edison schools and those in comparison 
schools was statistically significant in favor of Edison 
for about 38% of the reading outcomes examined for 
Edison. Thus, the rating in this subcategory is 
moderate. 

Evidence of Positive Effects in Subject Areas: Writing 
Rating: 0 

Two studies that met the CSRQ Center’s standards 
examined the TAAS to measure the writing achieve- 
ment of elementary school students. The first study 
was considered to be suggestive and reported a nega- 
tive trend. The second study was considered to be 
conclusive and reported a positive trend. But, neither 
study reported a level of statistical significance. Thus, 
the rating in this subcategory is zero. 

Evidence of Positive Effects in Subject Areas: Math 
Rating: 3 

Eive studies that met the CSRQ Center’s standards 
examined the impact of Edison on student achievement 
in math. Qf those, four studies were considered to be 



^This study defined low-achieving students as those who scored in the lowest 25% on the language proficiency test at the baseline year. 
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conclusive. One of these studies examined the math 
performance of elementary school students in a large 
school district in Texas and found that students in 
Edison schools did not meet or exceed the perform- 
ance of students in comparison schools. However, this 
study did not report a level of statistical significance 
among the findings. 

A second study reported no statistically significant 
effects of Edison on math achievement. 

A third study found statistically significant positive 
effects on math achievement of high school students, 
hut an effect size was not reported and could not he 
estimated hy the CSRQ Center. 

The fourth study found a statistically significant positive 
impact of Edison on math achievement of elementary 
school students. The CSRQ Center calculated an effect 
size of +0.34 for this positive outcome. 

Einally, the fifth study that met standards and was con- 
sidered to he suggestive did not provide information 
on the level of statistical significance of the outcomes 
examined. This study reported a mix of positive, null, 
and negative trends that varied hy school. 

Across these studies, the positive findings constituted 
about one half (49%) of the math outcomes with a 
reported level of statistical significance. Therefore, the 
rating for evidence of positive impact on math achieve- 
ment is moderate. 



E vidence of Positive Effects on 
Additional Outcomes 

Rating: 

No studies of Edison that examined additional out- 
comes were available for review. Therefore, the rating 
in this category is no rating. 

EVIDENCE OF POSITIVE EFFECTS ON STUDENT ACHIEVEMENT 



E vidence of Positive Effects on Parent, 
Eamily, and Community Involvement 

Rating: 

No studies that examined effects on parent, family, 
and community involvement were available for review. 
Therefore, the rating in this category is no rating. 

E vidence of Link Between Research and 
the Model’s Design 

Rating: ^ 

Based on documentation provided by Edison, explicit 
citations support all of the core components of the 
Whole School Management model: organization and 
governance; professional development; technical assis- 
tance; curriculum; instruction; inclusion; technology; 
time and scheduling; instructional grouping; student 
assessment; data-based decision making; and parent, 
family, and community involvement. Therefore, 
according to the CSRQ Center’s standards, the Whole 
School Management model rating for evidence of link 
between research and the model’s design is very strong. 

Eurthermore, based on documentation provided by 
Edison, explicit citations also support all of the core 
components of the Alliance model: professional devel- 
opment; technical assistance; inclusion; student assess- 
ment; data-based decision making; and parent, family, 
and community involvement. Therefore, according to 
the CSRQ Center’s standards, the Alliance model rating 
for evidence of link between research and the model’s 
design is very strong. 

Across both Edison models, the CSRQ Center’s rating 
for evidence of link between research and the model’s 
design is very strong. 
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E vidence of Services and Support to 
Schools to Enable Successful Implementation 

Evidence of Readiness for Successful Implementation 
Rating: # 

Based on documentation provided by Edison, the 
Whole School Management model and the Alliance 
model offer a formal process to help school staff estab- 
lish an initial understanding of the respective models.® 
Additionally, both models offer a formal process for 
allocating school resources — such as materials, staffing, 
and time — and provide formal benchmarks for imple- 
mentation. Therefore, according to the CSRQ Center’s 
standards, the Whole School Management model rating 
for evidence of readiness for successful implementation 
is very strong. Also, the Alliance model rating for 
evidence of readiness for successful implementation 
is very strong. 

Across both Edison models, the CSRQ Center’s rating 
in this category is very strong. 

Evidence of Professional Development/Technical 
Assistance for Successful Implementation 

Rating: • 

The Whole School Management model and the Alliance 
model provide ongoing training opportunities, such 
as workshops, peer coaching, capacity building, and 
sessions for new staff. Additionally, both of the models 
provide supporting materials for professional develop- 
ment that address all of their core components. Edison 
also offers a comprehensive plan to help build school 
capacity to provide professional development. Therefore, 
according to the CSRQ Center’s standards, the Whole 
School Management model rating for evidence of 
professional development/technical assistance for 



successful implementation is very strong. Also, the 
Alliance model rating for evidence of professional 
development/technical assistance for successful imple- 
mentation is very strong. 

Across both Edison models, the CSRQ Center’s rating 
in this category is very strong. 



CZ/ entral Components 



Administrative Services 

Eor schools that implement the Whole School 
Management model, Edison claims to provide “soup 
to nuts” administrative services and to assume man- 
agement and operational control of the school. Its 
administrative services include operation manage- 
ment, financial management, facilities management, 
and human resources management. Notably, two types 
of school partnerships fall under the Whole School 
Management model: district partnerships and Edison 
charter schools. The administrative services provided 
to a school depend on the type of partnership. 

In a district partnership, Edison assumes management 
and operational control of a school at the request of a 
district. Eurthermore, a district may hire Edison to 
assume responsibility for the management of the entire 
district. These schools receive financial management 
services — including budget oversight, payroll manage- 
ment, and audit assistance — from an Edison financial 
manager. In addition, Edison provides such human 
resources services as screening, interviewing, and hiring 
school staff (e.g., the school principal, administrative 
staff, and teachers). Teachers are offered opportunities 
for career advancement and increased pay through a 
career ladder that moves from resident teacher to lead 
teacher. The various teaching positions are described 
in greater detail in “Qrganization and Governance.” 



*As noted in “Methodology,” the following subcategory, which has been used to rate school reform models in previous reports from the CSRQ Center, is not 
applicable to ESPs: Provider ensures initial commitment from schools. 
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Edison charter schools receive the same financial and 
human resources services hut they also receive addi- 
tional administrative support. In the start-up process, 
either during the application process or after the charter 
is granted, Edison partners with the nonprofit organi- 
zation or group (also known as the charter hoard) that 
is applying for the charter. If needed, Edison provides 
assistance with the charter application. After the charter 
is granted, a general manager from Edison works with 
the charter hoard, school staff, and principal to support 
the start-up process of the charter school. To this end, 
Edison provides facilities management services, such as 
identifying and acquiring a school site and acquiring 
funding for the site. In addition, Edison helps charter 
schools to recruit new students and maintain student 
enrollment through a community outreach and market- 
ing plan. All Edison charter schools are public schools; 
therefore, they are open to all students. 

Conversely, Edison Alliance schools do not receive 
administrative services. These schools remain under 
the management and operational control of the district. 

Organization and Governance 

The organizational structure of the Edison school design 
is intended to create learning communities in which all 
students are known as individuals. In the Whole School 
Management model, each Edison school is designed 
to have smaller, flexible schools within schools, also 
known as academies. Eive types of academies serve 
grades K-12: 

■ Primary Academies (K-2) 

■ Elementary Academies (3-5) 

■ Junior Academies (6-8) 

■ Senior Academies (9-10) 

■ Collegiate Academies (11-12) 



Within each academy, students are placed into multi- 
grade “houses” of 100-180 students. A team of four to 
six teachers, directed by a lead teacher, is assigned to 
each house. The team stays with this group of students 
(or house) for their entire experience in the academy. 

According to Edison, the success of this organizational 
structure depends upon strong academic leadership 
from teachers, the school principal, and the school 
leadership team. All teachers receive opportunities for 
leadership and career advancement. Teachers who have 
recently completed a teacher preparatory program join 
Edison as a resident teacher. These teachers work under 
a lead teacher to improve their instructional skills and 
develop long-range professional development goals. 
With 2 years of teaching experience and teacher certi- 
fication, a teacher is given the responsibility of writing 
curricula, communicating with parents, and developing 
professional relationships with other Edison staff. After 
years of experience as a teacher and proven mastery in 
the areas of curriculum, instructional methods, and 
classroom management, teachers are given the title of 
senior teacher and are appointed as curriculum coor- 
dinators. Senior teachers are expected to administer 
assessments, model instructional methods to other 
teachers, and provide regular updates to the school 
principal. Senior teachers who have master’s degrees 
(or are pursuing master’s degrees); have 3 to 5 years of 
teaching experience; have experience creating and pub- 
lishing units, assessments, and other Edison curriculum 
materials; and exhibit success in the classroom are asked 
to be lead teachers. Lead teachers are the instructional 
and organizational leaders of the houses within each 
academy. Edison provides all teachers with explicit 
benchmarks and indicators for the organization of 
each house. 

Likewise, the school principal supports all teachers as 
an instructional leader, site manager, and “builder of 
school culture.” As an instructional leader, the school 
principal monitors the implementation of curricular 
programs, supervises and evaluates staff, and helps 
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create professional growth plans for teachers and other 
staff. As the site manager, the school principal manages 
school operations, creates emergency and safety policies, 
manages school resources, creates human resources 
policies, and monitors budding maintenance. As the 
builder of school culture, the school principal creates 
goals for working toward the model’s mission, encour- 
ages teamwork through houses, monitors technology, 
and develops relationships with parents and community 
members. Edison holds all principals accountable in 
five areas: student achievement, school design, customer 
satisfaction, financial management, and Edison system 
growth (i.e., expanding the national Edison network). 
Each principal receives explicit benchmarks and indi- 
cators to track his/her growth as a school principal. 

The school leadership team is appointed by the school 
principal. The leadership team collaborates with the 
school staff, principal, and houses to monitor student 
achievement and the school design. To this end, the 
team monitors and analyzes achievement data to ensure 
that intervention strategies are being used appropriately 
and meets regularly with the curriculum coordinators 
or senior teachers to review and revise instructional 
strategies. The leadership team also works closely with 
the school principal to set goals, implement curricular 
programs, and develop relationships with parents and 
community members. Edison provides the leadership 
team with explicit benchmarks and indicators for 
carrying out these tasks. 

These organizational and leadership structures are 
recommended to Edison Alliance schools but organi- 
zation and governance solutions are customized to 
meet the needs of each school. 

Curriculum and Instruction 

The Edison school design includes curricular programs, 
either selected or developed by Edison, for all core 



content areas. The curriculum covers five domains: 
math and science, humanities and the arts, character 
and ethics, practical arts and skills, and physical fitness 
and health. The curricular program and academic 
standards for each of these domains is specific to each 
academy. 

In the Primary and Elementary Academies (K-5), 
schools are required to adopt either Success for AU 
(http://www.successforall.net/)^ or Open Court 
(http://www.sraonline.com) as their reading curriculum 
and to deliver 90 minutes of reading instruction daily 
using one of these curricular programs. Both curricular 
programs include a mandatory tutoring program. 
Likewise, schools are required to adopt the University 
of Chicago School Mathematics Project’s Everyday 
Mathematics (http://everydaymath.uchicago.edu/) and 
to deliver 60 minutes of math instruction daily. The 
Edison school design also requires schools to adopt its 
science, social studies, and writing/language arts curric- 
ula and to deliver 45 minutes of daily instruction in each 
of these content areas. Several times a month, teachers 
are required to use the social studies instructional block 
to teach a character and ethics lesson. The Edison 
character and ethics curriculum was developed by the 
Heartwood Institute (http://www.heartwoodethics.org/). 
In addition, the daily instructional schedule includes 
45 minutes for physical education (PE), art, music, or 
world language. 

In the Junior Academy (6-8), schools are required to 
offer 90 minutes of uninterrupted reading/language arts 
instruction. Ml students receive the Scholastic Reading 
Inventory (http://teacher.scholastic.com/products/sri/), 
and if students score below grade level, they are given 
the Woodcock Reading Mastery test to determine 
whether a decoding problem exits. If this problem 
exists, the student is placed in Wilson Reading 
(http://www.wilsonlanguage.com/). Wilson Reading is 
taught during the regularly scheduled reading block. 



’^The CSRQ Center reviewed the Success for All model in an earlier report: CSRQ Center Report on Elementary School Comprehensive School Reform Models. 
The report can be accessed at http://www.csrq.org. 
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Schools are also required to offer 45 minutes of math 
instruction daily. Students are placed in the appropriate 
math class based upon their math background and 
skill level. Schools are required to offer the following 
math courses: Everyday Math (grade 6), Transitions 
(grades 7 and 8), Extended Transitions (grades 7 and 
8), and Contemporary Mathematics in Context 
(http://www.glencoe.com/sec/math/cpmp/). 
Eurthermore, schools are required to adopt the Edison 
social studies curriculum and a science curriculum, 
which was designed by Biological Sciences Curriculum 
Study (http://www.bscs.org/). Students receive 45 min- 
utes of social studies and science instruction daily, in 
addition to 45 minutes of world language, PE, and 
fine arts. 

In the Senior Academy (9-10), schools are required to 
provide 65 minutes of instruction in math (algebra, 
geometry, calculus), literature and language arts (world, 
American, and British literature), science (biology, 
chemistry, and physics), social studies (courses based 
on state requirements), world language (conversational 
Spanish), fine arts, and PE. AU curricula are designed 
to prepare students for Advanced Placement (AP) 
courses offered in the Collegiate Academy (10-12). 

In the Collegiate Academy, schools are required to 
offer 65 minutes of instruction in math, literature and 
language arts, science, social studies, world language, 
fine arts, and PE. However, in the Collegiate Academy, 
course selection is expanded to include multiple AP 
courses in each content area as well as dedicated time 
for college preparation. Edison partners with Princeton 
Review to provide students with orientation to college 
applications, SAT or ACT preparation, and career 
counseling. 

In all five academies, teachers are required to deliver 
curricula using specific instructional strategies. These 
strategies include project-based learning, direct 
instruction, cooperative learning, and differentiated 
instruction. Teachers receive training on each of these 
instructional strategies and explicit benchmarks that 



guide the implementation of the strategy. The school 
leadership team and the curriculum coordinator sup- 
port teachers by modeling these strategies and moni- 
toring implementation. Eurthermore, the Edison school 
design embraces a policy of responsible inclusion — 
a commitment to include all students, including those 
with disabilities, to the extent possible, in the general 
education classroom. To this end, general and special 
education teachers are trained in the instructional 
methods of co-planning and co-teaching. 

Schools that adopt the Alliance model have the flexi- 
bility to choose their own curricular programs but are 
trained to implement the aforementioned instructional 
strategies and are required to adopt the policy of 
responsible inclusion. 

Scheduling and Grouping 

The Edison school design seeks to organize the school 
day so that instructional time increases and teachers 
have ample time for planning and professional devel- 
opment. Although daily schedules vary by school, each 
school that adopts the Whole School Management 
model is required to implement the scheduling guide- 
lines described previously in “Curriculum and 
Instruction.” During the school start-up process or 
planning stage, an Edison scheduling specialist works 
with school staff to create a master schedule based upon 
the length of the school day. Although Edison Alliance 
schools are not required to adhere to strict scheduling 
guidelines, they also receive these consultative services. 

Edison has four fundamental policies for designing a 
school schedule: 

■ All schedules must adhere to the curricular and 
instructional school design. 

■ The schedule must accommodate smaller class sizes 
and include dedicated instructional blocks, houses, 
and access to special subjects (e.g., language, fine 
arts, PE). 
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■ The schedule must allow for common planning 
time within houses, morning meetings (K-5), 
home-hased advisory (6-8), and midday meetings 
(9-12). 

■ The schedule must increase time for instruction. 

Edison strongly recommends that schools adopt an 
extended school day and school year. According to the 
Edison school design, the recommended length of a 
school day for the Primary and Elementary Academies 
is 7 hours and 8 hours for the Junior, Senior, and 
Collegiate Academies. Edison also recommends 
extending the school year from 180 to 200 days. 

As described in “Organization and Governance,” 
students are heterogeneously grouped in multigrade 
houses. In the Primary, Elementary, and Junior 
Academies, students are taught reading and math in 
homogenous groups and all other subjects in hetero- 
geneous groups. In the Senior and Collegiate 
Academies, students select courses based on back- 
ground and skill level. 

All scheduling and grouping strategies are recom- 
mended for Edison Alliance schools but these compo- 
nents are customized to meet the needs of each school. 

Technology 

Schools that implement the Whole School Management 
model are required to use technology as both an 
instructional and management tool. Rather than using 
technology to replace instruction, Edison expects 
teachers to use it as tool to improve communication, 
research, writing, and analysis. All teachers and stu- 
dents have access to technology in their classroom. 
Specifically, each classroom is equipped with computers, 
televisions, and VCRs. In addition, most schools have 
wireless labs that travel between classrooms. Wireless 
labs are used for computer-based benchmark assess- 
ments, which are described in “Monitoring Student 
Progress and Performance.” 



In addition, each teacher is given a laptop computer for 
the academic year to be used for research and lesson 
planning, maintaining an online grade book, and 
reviewing benchmark assessments. If needed, students 
in grades 3 and higher are also given a laptop computer 
for home use during the academic year. Edison staff 
support schools in the implementation of technology, 
including allocating funding for hardware and software 
purchases and networking. 

Edison Alliance schools are not required to use tech- 
nology as an instructional tool but are required to use 
online benchmark assessments. 

Monitoring Student Progress and Performance 

Student and teacher accountability is essential to the 
Edison school design. Eor this reason, the Edison 
school design includes four types of student assess- 
ments: state and district assessments, benchmark 
assessments, structured portfolios, and quarterly 
learning contracts. Edison expects all assessments to be 
stored electronically so that the school principal and 
leadership team can monitor student performance. 

■ State and district assessments. The Edison staff, 
school leadership team, and the school principal 
work together to set achievable goals on state and 
district assessments. In addition, Edison seeks to 
align curricular programs with state and district 
standards. 

■ Edison Benchmark Assessment System. Each 
school that adopts the Whole School Management 
model or the Alliance model is required to use the 
diagnostic and assessment tools designed by 
Tungsten Learning, a division of Edison Schools. 
These tools are packaged in an online benchmark 
assessment system. The system includes norm- 
referenced tests that cover academic standards in 
reading, writing/language arts, and math for grades 
2-10. These tests are designed to be administered 
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monthly. The system also provides teaching notes 
that help teachers to modify instructional strategies 
based on assessment results. In addition, the system 
allows teachers and administrators to track and 
graph student data. Administrators also have the 
option to create real-time reports of aggregated 
and disaggregated student data. 

■ Structured portfolio. Teachers maintain an elec- 
tronic portfolio of each student’s work. Each quarter, 
teachers and students jointly select a few pieces of 
work that illustrate a student’s improvement over the 
quarter. Edison provides suggestions for portfolio 
entries. 

■ Quarterly learning contracts. Each quarter, Edison 
plans family conferences that involve the student, 
teacher, and parents. During these quarterly con- 
ferences, the group develops a quarterly learning 
contract that provides an overview of the student’s 
performance during the past quarter; establishes 
goals for the upcoming quarter; and requires the 
signature of the student, teacher, and parents. Edison 
recommends that schools allocate 2 half days per 
quarter for teachers to prepare quarterly learning 
contracts and to conduct family conferences. 

Family and Community Involvement 

The Whole School Management model and the 
Alliance model require the involvement of family 
members. As described in “Monitoring Student 
Progress and Performance,” parents are required to 
attend quarterly conferences where learning contracts 
are developed. These contracts are designed to involve 
parents in the learning process and to keep them 
informed about their child’s progress toward end-of- 
year goals. In addition, each school has a Eamily and 
Student Support Team that works with families who 
are experiencing problems that may interfere with 
their child’s academic performance. Schools also have 
a Parent Advisory Council that is used to elicit the 



support of parents for school programs and practices. 
Einally, Edison encourages parent volunteerism. 

Professional Development and Technical Assistance 

The Edison school design requires schools to create 
professional learning environments for all staff. Edison 
supports schools that implement both the Whole School 
Management model and the Alliance model by offering 
intensive leadership training, pre-implementation cur- 
riculum and instructional training for staff, and ongo- 
ing professional development and technical assistance 
during implementation. The professional development 
opportunities are offered through onsite workshops, 
national and regional conferences, onsite mentoring 
and monitoring, and online and teleconference training 
sessions. 

The spring before implementation begins, Edison pro- 
vides Edison Leadership Team Training (LETT). At 
LETT, Edison provides workshops on student achieve- 
ment, student management, and leadership and design 
implementation to new principals, the school leadership 
team, and new lead teachers. These workshops are 
generally offsite and last 4 days. Eor more experienced 
principals and school leadership teams, Edison provides 
ongoing training and support through the Edison 
Leadership Development Academy (ELDA). The 
academy begins with an intensive summer conference, 
which lasts 4 days, and continues throughout the year 
with discussion groups, teleconference training sessions, 
and regular mentoring from ELDA’s facilitators. 

The summer before implementation, all new profes- 
sional staff members attend the Edison Teaching 
Academy, also known as Camp Edison, for an intensive 
4-day, offsite training on technology, curriculum, 
instruction, and classroom management. Teachers also 
receive onsite training on the Edison Benchmark 
Assessment System that lasts 1 day. 
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In addition, school staff members receive ongoing 
professional development and technical assistance 
from an Edison support team. This support team is 
comprised of reading and math specialists, technology 
specialists, and advisors that provide expertise in 
financial management, scheduling, English language 
learners, and classroom management. This ongoing 
support is provided through onsite meetings, phone 
conferences, and classroom observations. Throughout 
the school year, school staff members are expected to 
attend a 4-day curriculum training, a 2-day learning 
environment training, and a 2-day benchmark training. 
Moreover, schools are required to build common plan- 
ning time and two professional development periods 
into the daily schedule for teachers. 

Implementation Expectations/Benchmarks 

Eor schools that implement the Whole School 
Management model and the Alliance model, Edison 
begins the partnership by conducting a diagnostic 
assessment to identify the school’s strengths and weak- 
nesses. The results of this assessment are used by 
Edison staff, in collaboration with the school principal, 
to create a Plan for Success. Each school that imple- 
ments the Whole School Management model is given a 
formal set of benchmarks and indicators to guide the 
implementation of this plan. Although Edison Alliance 
schools have access to these benchmarks, their plan 
and implementation goals are customized to meet the 
needs of each partnership school. 

Special Considerations 

The services that a school receives from Edison depend 
on the Edison model a school selects. The Whole 
School Management model closely adheres to the 
Edison school design; the Alliance model is more 
dependent upon the needs of each school. Nonetheless, 
charter authorizers contacted by the CSRQ Center 
noted that Edison is willing to tailor the Whole School 



Management model to meet the needs of the local 
community and to accommodate budget limitations. 
Ml districts, charter authorizers, and school principals 
contacted by the CSRQ Center emphasized that a 
strong commitment to the model is needed for the 
change process to be successful. 



M odel Studies Reviewed 

Met Standards (Suggestive) 

American Eederation of Teachers. (1998). Student 
achievement in Edison Schools: Mixed results in 
an ongoing enterprise. Washington, DC: Author. 

Miron, G., & Applegate, B. (2000). An evaluation of 
student achievement in Edison schools opened in 
1995 and 1996. Kalamazoo, MI: The Evaluation 
Center, Western Michigan University. 

Met Standards (Conclusive) 

Dryden, M. (2004). The performance of Edison Schools 
Inc. in the Dallas schools. Paper presented at the 
Annual Meeting of the American Educational 
Research Association, San Diego, CA. 

Gill, B. P, Hamilton, L. S., Lockwood, J. R., Marsh, J. 
A., Zimmer, R. W, Hill, D., et al. (2005). 
Inspiration, perspiration, and time: Operations 
and achievement in Edison Schools. Arlington, 
VA: RAND Gorporation. 

Miron, G. (2006). Evaluation of the Delaware Charter 
school reform, year 2 report. Kalamazoo, MI: 

The Evaluation Genter, Western Michigan 
University. Retrieved Eebruary 21, 2006, 
from http://www.doe.kl2.de.us/files/pdf/ 
dedoe_charterschreform2006.pdf 

Mislevy, R. J. (1996). Reading achievement test-score 
analysis: 1995/96 Washington-Edison School, 



CENTRAL COMPONENTS 



46 




EDISON SCHOOLS 



grades K-2 Sherman Independent School 
District. Retrieved January 24, 2006, from 
http://web.archive.Org/web/19990209120006/ 
www.aft.org/research/edisonproject/sfa/w96/ 
mislevy.htm 

Mislevy, R. J. (1997). Reading achievement test-score 
analysis: 1 996197 Dodge-Edison vs. control 
schools, grades 1 -3 Wichita Unified School 
District #259. Retrieved January 24, 2006, from 
http://web.archive.Org/web/19990209120006/ 
www.aft.org/research/edisonproject/sfa/w96/ 
mislevy.htm 

Mislevy, R. J. (1997). Reading achievement test-score 
analysis, 1 996/97, grades K-2, Roosevelt- Edison 
School, Colorado Springs, CO. Retrieved January 
24, 2006, from http://web.archive.org/web/ 
19990209120006/www.aft.org/research/ 
edisonproject/sfa/w96/mislevy.htm 




Shay, S. A., & Gomez, J. J. (2002). Privatization in 

education: A growth curve analysis of achievement. 
Paper presented at the Annual Meeting of the 
American Educational Research Association, 
New Orleans, LA. 
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Imagine Schools 



Overview: 


Basic Model Information and Review Results 


Model Name: 


Imagine Schools 


Model Mission/Focus: 


Imagine Schools’ philosophy emphasizes a challenging, effective academic program, 
strong character formation, solid financial health for each school, and a positive school 
culture built on adherence to the shared principles of integrity, justice, and fun. 


Year Introduced in Schools: 


2004 


Grade Levels Served: 


Pre-K-12 


Number of Schools 

Total: 

42 


Urban: Suburban: Rural: 

N/A N/A N/A 



Costs 


Total Operating 
Costs: 


Training: 


Materials: 


Personnel: 


Other: 


Yearl 


N/A 


N/A 


N/A 


N/A 


N/A 


Year 2 


N/A 


N/A 


N/A 


N/A 


N/A 


Year 3 


N/A 


N/A 


N/A 


N/A 


N/A 


Years 4+ 


N/A 


N/A 


N/A 


N/A 


N/A 



1 . Evidence of Positive Effects on Student Achievement: 

a. Overall effects ® 

b. Evidence of posifive effecfs for diverse studenf populations ® 

c. Evidence of posifive effecfs In subject areas ® 

2. Evidence of Positive Effects on Additional Outcomes ® 

3. Evidence of Positive Effects on Parent, Family, and Community Involvement ® 

4. Evidence of Link Between Research and the Model’s Design ® 

5. Evidence of Services and Support to Schools to Enable Successful Implementation: 

a. Evidence of readiness for successful implementation ® 

b. Evidence of professional developmenf/fechnical assisfance for successful implemenfafion ® 



^ = Very Strong ^ = Moderately Strong 3 = Moderate = Limited 0 = Zero Q = Negative @ = No Rating 



This description is based on publicly available information, including the model’s Web site, regarding the model and its 
costs. The Comprehensive School Reform Quality Center attempted to obtain specific information, but this was not always 
possible. Areas in which exact information was not provided are marked by “N/A.” 



IMAGINE SCHOOLS 



M odel Description 

The information presented in the following description 
of Imagine Schools was collected using (a) Imagines 
Web site and (b) conversations with three charter 
authorizers that oversee the charter authorization 
process for schools that use this model. Although 
contact was initiated with a representative of Imagine, 
the Comprehensive School Reform Quality (CSRQ) 
Center was unable to conduct a conversation with the 
education service provider (ESP). 

Imagine Schools, an employee-owned company, estab- 
lished itself as an ESP in 2004 when Eileen and Dennis 
Bakke bought out Chancellor Beacon Academies, 
another charter management company.^ Imagine offers 
charter management services, including financial 
management and human resources, to both existing 
charter schools and community members that are 
interested in starting new charter schools. 

According to the CSRQ Center’s standards, the follow- 
ing were identified as core components of Imagine: 
curriculum; instruction; student assessment; data-based 
decision making; and parent, family, and community 
involvement. Core components are considered essential 
to successful implementation of the model. 

Mission/Focus 

According to the provider’s Web site, the philosophy 
of Imagine is to emphasize a challenging and effective 
academic program; strong character formation; solid 
financial health for each school; and a positive school 
culture built on adherence to the shared principles of 
integrity, justice, and fun. To accomplish this mission, 
the provider focuses on four main components for 
successful learning: 



■ Qualified teachers committed to implementing the 
school’s mission and educational approach 

■ A curriculum that stresses mastery of the basic build- 
ing blocks of learning while providing subject matter 
that is rich and varied and engages students’ curiosity 

■ A school environment that is safe, orderly, purpose- 
ful, and positive 

■ The involvement of parents in an active role that is 
aligned with the educational mission of the school 

Goals/Rationale 

The provider’s Web site indicates that the founders’ 
goal is to make a difference in education by imple- 
menting a unique organizational and operational 
structure that puts teachers and school leaders squarely 
in charge of the decisions that affect the schools they 
serve. To accomplish this goal. Imagine allows each 
school to select a structure that is based on the specific 
needs of the school community. 

Although the provider emphasizes site-based control 
for almost all aspects of the school. Imagine stresses 
the importance of Professor Ronald R. Edmonds’s 
Effective Schools Research to create a healthy learning 
environment. (Mr. Edmonds is the former director of 
the Center for Urban Studies at Harvard University.) 
Imagine underscores several aspects of Effective Schools 
Research: a clear school mission, instructional leader- 
ship, a home and school partnership, high expectations, 
the opportunity to learn, a safe and orderly environ- 
ment, and frequent monitoring of student progress. 



C osts 

Eor more information on the costs of training, materials, 
and personnel, sites should directly contact the ESP. 



^In August 2005, Imagine Schools created a new nonprofit organization, Imagine Schools Non-Profit, Inc., which intends to purchase the ownership interests 
of Imagine Schools, Inc. Operations for its existing charter schools will then be conducted as a nonprofit educational service provider. 
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E vidence of Positive Effects on Student 
Achievement 

Evidence of Positive Overall Effects 
Rating: ® 

No quantitative studies that examined the effects of 
Imagine on student achievement were available for 
review by the CSRQ Center. Therefore, the overall 
rating of the evidence of positive overall effects of this 
model on student achievement is no rating. 

Evidence of Positive Effects for Diverse Student 
Populations 

Rating: ® 

Because no studies of Imagine were available for review 
by the CSRQ Center, the impact of this model on 
student achievement for diverse student populations 
is unknown. Therefore, the rating in this subcategory 
is no rating. 

Evidence of Positive Effects in Subject Areas 
Rating: ® 

Because no studies of Imagine were available for review 
by the CSRQ Center, the rating in this subcategory is 
no rating. 

E vidence of Positive Effects on 
Additional Outcomes 

Rating: @ 

Because no studies of Imagine were available for review 
by the CSRQ Center, it was not able to evaluate the 
effects of Imagine on additional outcomes. Therefore, 
the rating in this category is no rating. 



EVIDENCE OF POSITIVE EFFECTS ON STUDENT ACHIEVEMENT 



E vidence of Positive Effects on Parent, 
Eamily, and Community Involvement 

Rating: @ 

Because no studies of Imagine were available for review 
by the CSRQ Center, the effects of Imagine on this 
category are unknown. Therefore, the rating in this 
category is no rating. 

E vidence of Link Between Research and 
the Model’s Design 

Rating: @ 

The CSRQ Center did not conduct a conversation 
with the ESP, nor was it able to find publicly available 
information to allow it to rate this category. Therefore, 
the rating in this category is no rating. 

E vidence of Services and Support to 
Schools to Enable Successful Implementation 

Evidence of Readiness for Successful Implementation 
Rating: ® 

The CSRQ Center did not conduct a conversation 
with the ESP. Therefore, it was not able to collect the 
evidence necessary to rate this subcategory. Therefore, 
the rating in this subcategory is no rating. 

Evidence of Professional Development/Technical 
Assistance for Successful Implementation 

Rating: ® 

The CSRQ Center did not conduct a conversation 
with the ESP, nor was it able to find publicly available 
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information to allow it to rate this subcategory. 
Therefore, the rating in this subcategory is no rating. 



CZ/ entral Components 



Administrative Services 

As described on its Web site and in publicly available 
information, Imagine provides a wide array of admin- 
istrative services to its schools. Services appear to 
include financial management, facilities management, 
and human resources management. The provider 
works with both existing schools and start-up schools 
to provide administrative services that are tailored to 
the school’s needs. 

The provider established an admissions process for 
students interested in attending a school managed by 
Imagine. Interested students must complete an appli- 
cation, submit recommendations, attend an informal 
meeting with the director of admissions, and take an 
aptitude test. When the amount of approved applica- 
tions exceeds the number of open spots at a school 
site, that school uses a lottery system to select students 
for admittance. 

Organization and Governance 

The overall organization and governance of individual 
schools served by Imagine is similar to the features 
found in most charter schools. Each Imagine school is 
likely to have a board of directors that oversees its 
progress, in addition to an outside entity that author- 
izes its charter and monitors its progress. States have 
different requirements for authorizers, but generally 
authorizers include local school districts and colleges 
or universities. Depending on state law, the employees 
of each Imagine school may be either employees of the 
school district or employees of Imagine. 



Because the provider emphasizes a model of site-based 
control, the organizational structure of each school may 
differ. However, Imagine encourages each school’s staff 
to commit themselves to a specific school mission that 
is focused on instruction and accountability. Imagine 
also highlights the importance of the principal as the 
instructional leader, who communicates this mission 
to the staff, parents, and students of the school. 

The provider’s Web site explains that to create a safe 
and orderly environment free from threats of physical 
harm. Imagine institutes a clear and consistent disci- 
pline policy. In addition, to help ensure a focus on 
learning, schools have either student uniforms or 
foUow a dress code. 

Curriculum and Instruction 

Imagine emphasizes instruction as one of the four main 
components for successful schools. However, given that 
the emphasis is on site-based control, the provider 
does not endorse or require one specific curriculum. 
For example, one charter authorizer explained that it 
develops the curriculum for the Imagine schools that 
it authorizes. The provider, therefore, does not have 
strict criteria for the curriculum used in its schools. 
Instead, the provider indicates general guidelines that 
schools should use in selecting curricula for two dif- 
ferent grade-level tracks: K-8 and 9-12. 

For grades K-8, according to the provider’s Web site, 
a school’s educational approach should focus on basic 
skills: reading, fluency, oral communication, and 
computation. Character education is also integrated 
into all classes and subjects. The primary grades focus 
on a phonics-based reading program, and as the grades 
progress, an increased focus in math, science, and his- 
tory emphasizes sequential learning, mastery of critical 
skills, and project-based instruction. In addition. 
Imagine offers a variety of literature, art, music, and 
drama components. 



EVIDENCE OF SERVICES AND SUPPORT TO SCHOOLS TO ENABLE SUCCESSFUL IMPLEMENTATION 
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For grades 9-12, according to the provider’s Web site, 
the high schools should focus on dropout prevention 
and workforce preparation or higher education. The 
curriculum for grades 9 and 10 should emphasize such 
core academic subjects as English, math, science, history, 
and foreign language. In grades 11 and 12, students 
select one of three academies in which to specialize: 
Humanities and the Arts, Business, or Science and 
Technology. As is found in the earlier grades, character 
education is emphasized in the different classes and 
subjects. 

Scheduling and Grouping 

Because each school develops its own structure. Imagine 
does not have specific scheduling or grouping require- 
ments for its schools. 

Technology 

Imagine does not specifically emphasize the use of tech- 
nology across all grade levels and schools, but high 
school students in grades 11 and 12 can elect to focus 
on technology in the Science and Technology Academy. 
For more information on the Science and Technology 
Academy, sites should directly contact the ESP. 

Monitoring Student Progress and Performance 

Imagine suggests that its schools use frequent assess- 
ments of student progress to determine individual 
student progress and to improve the instructional 
program. In addition to such diagnostic assessments 
as the Stanford Achievement Test, Ninth Edition and 
the Dynamic Indicators of Basic Early Literacy Skills, 
each school follows specific state testing requirements. 

Family and Community Involvement 

Imagine notes that an essential component for success- 
ful implementation is the involvement of parents in 



the school’s educational mission. Each school asks 
parents to take an active role in their child’s education 
by signing a covenant with their child that details 
expectations for students, parents, and the school. 

Professional Development and Technical Assistance 

The provider’s Web site mentions that Imagine provides 
teachers and counselors with professional development 
tools through The Imagine Schools Institute. However, 
the provider’s emphasis on site-based control also 
allows different professional development plans for 
each school site. For example, one charter authorizer 
of an Imagine school noted in a conversation with the 
CSRQ Center that within one school district containing 
several Imagine schools, the schools have implemented 
a “school of excellence” program in which teachers and 
principals visit other Imagine sites to offer suggestions 
for improvement and to observe successful aspects of 
their program. 

Implementation Expectations/Benchmarks 

The provider’s Web site indicates that Imagine reviews 
implementation progress for its schools on an annual 
basis. For some schools, progress may be monitored 
on a monthly basis. For more specific information on 
timelines and benchmarks for implementation, schools 
should directly contact the ESP. 

Special Considerations 

According to its Web site, the provider stresses the 
importance of site-based control so that each school 
can tailor educational services to the school’s individual 
needs. However, one authorizer noted that when 
Imagine took over for Chancellor Beacon, the 1st year 
was a challenging transition for teachers and staff who 
were suddenly given control over many aspects with- 
out proper training by the provider. However, this same 
authorizer noted that, after the 1st year, the site-based 
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control feature became one of the most attractive 
components to many teachers. 

One charter authorizer noted that Imagine offers a 
flexible human resources operation that offers a desir- 
able alternative to traditional teacher hiring and salary 
schedules. 

Although the costs for this model are not publicly 
available, one authorizer noted that the large manage- 
ment fee would not be economically sound for smaller 
charter schools (e.g., with 200 to 400 students). 
However, this authorizer strongly recommended this 
provider for larger charter schools (e.g.., schools with 
more than 500 students). 

Readers should note that the CSRQ Center was unable 
to conduct a conversation with the ESR The description 
of Imagine was based on publicly available information 
and conversations with three charter authorizers. For 
more specific information on Imagine, schools should 
directly contact the ESR 
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The Leona Group, L.L.C. 



Overview: Basic Model Information and Review Results 

Model Name: The Leona Group, L.L.C. 



Model Mission/Focus: According to the model’s Web site, the mission of The Leona Group, L.L.C. is to 

encourage choice and competition within America’s public education system to improve 
the academic achievement of all students, including those with special needs. 



Year Introduced in Schools: 


1996 








Grade Levels Served: 


Pre-K-12 








Number of Schools 










Total: 




Urban: 


Suburban: 


Rural: 
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N/A 


N/A 


N/A 




Costs 


Total Operating 
Costs: 


Training: 


Materials: 


Personnel: 


Other: 


Yearl 


Varies 


Varies 


Varies 


Varies 


Varies 


Year 2 


Varies 


Varies 


Varies 


Varies 


Varies 


Year 3 


Varies 


Varies 


Varies 


Varies 


Varies 


Years 4+ 


Varies 


Varies 


Varies 


Varies 


Varies 



1 . Evidence of Positive Effects on Student Achievement: 

a. Overall effects 0 

b. Evidence of positive effects for diverse student populations ® 

c. Evidence of positive effects in subject areas ® 

2. Evidence of Positive Effects on Additional Outcomes ® 

3. Evidence of Positive Effects on Parent, Family, and Community Involvement ® 

4. Evidence of Link Between Research and the Model’s Design Q 

5. Evidence of Services and Support to Schools to Enable Successful Implementation: 

a. Evidence of readiness for successful implementation 3 

b. Evidence of professional development/technical assistance for successful implementation 3 



3 = Vs'y strong 3 = Moderately Strong 3 = Moderate = Limited 0 = Zero 0 = Negative @ = No Rating 



This description is based on publicly available information, including the model’s Web site, regarding the model and its 
costs. The Comprehensive School Reform Quality Center attempted to obtain specific information, but this was not always 
possible. Areas in which exact information was not provided are marked by “N/A.” 



THE LEONA GROUP, L.L.C. 



M odel Description 

In 1996, Dr. William Coats formed The Leona Group, 
L.L.C. — a privately held education service provider 
(ESP) — to develop and manage public charter schools 
in the Midwest. In its 1st year, Leona was hired hy the 
board of directors of four charter schools in Michigan 
to provide start-up and management services. These 
services included, but were not limited to, human 
resources, educational programming, financial manage- 
ment, communications and marketing, and technology 
consulting. During the past decade, Leona has been 
hired to provide these services to more than 3 1 schools 
in Indiana, Michigan, and Ohio. 

In 1997, Leona opened a second division in Arizona. 
Unlike states in the Midwest, the charter authorizer in 
Arizona issues contracts directly to managing organi- 
zations. Therefore, Leona owns and operates the con- 
tract for a group of 16 charter schools in Arizona. In 
both the Arizona and Midwest divisions, Leona seeks 
to serve poor and minority students in urban areas. 

According to the CSRQ Center’s standards, the follow- 
ing were identified as core components of Leona: 
organization and governance; professional development; 
technical assistance; inclusion; and technology. Core 
components are considered essential to successful 
implementation of the model. 

Model Mission/Focus 

According to the model’s Web site, Leona’s mission is 
to encourage choice and competition within America’s 
public education system to improve the academic 
achievement of all students, including those with special 
needs. To achieve this mission, Leona seeks to provide 
comprehensive management services to public charter 
schools in Arizona and the Midwest. Although each 
Leona-managed school uses a customized curricular 



and instructional program, these schools share common 
elements: 

■ Inclusive classrooms and personal learning pro- 
grams for each student. Leona-managed schools 
embrace a philosophy of full inclusion. All special 
education students are taught in general education 
classrooms, and teachers develop a personal learning 
program for all students. 

■ A safe and secure environment. Leona-managed 
schools develop codes of conduct and an action 
plan for emergency situations to ensure that all 
schools are safe. 

■ Strong ties between home and school. Leona- 
managed schools encourage parental and commu- 
nity involvement through volunteerism, planning 
committees, and parent-teacher associations. In 
addition, Leona-managed schools often provide 
before and after school care. 

■ Flexible scheduling. School leaders work with 
Leona to develop a customized daily schedule and 
school calendar. 

■ A qualified staff that is screened and employed 
by Leona. Leona seeks to hire, train, and pay 
qualified school leaders, teachers, and support 
staff at all Leona schools. 

Goals/Rationale 

Leona believes that each school has unique character- 
istics that are based on (a) the community that the 
school serves and (b) the needs of the school’s students 
and families. Therefore, Leona provides schools with 
whole school management and start-up services 
(financial, human resources, marketing, technology 
consulting, etc.) and works closely with parents, school 
improvement teams, and school staff to develop a 
customized educational program that includes cur- 
riculum, instruction, scheduling, and assessment. 
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The guiding principle for all educational programs at 
Leona-managed schools is that all students will succeed 
if they are held to high expectations. For this reason, 
all students, including students with special needs, are 
educated in general education classrooms with peers 
of the same age. To ensure that all students succeed in 
the general education classroom, teachers are expected 
to modify instructional strategies and classroom activ- 
ities to meet the needs of all learners. 



C osts 

The costs for the services provided by Leona typically 
equal a contracted fee that represents a percentage of 
each charter school’s budget. This fee varies by school 
and state. For more information on the costs of training, 
materials, and personnel, sites should directly contact 
the ESP. 



E vidence of Positive Effects on Student 
Achievement 

Evidence of Positive Overall Effects 
Rating: 0 

The CSRQ Center reviewed two quantitative studies 
of Leona on student achievement. Neither study had 
sufficient rigor to meet the CSRQ Center’s standards. 
Therefore, the overall rating of the evidence of positive 
overall effects of this model on student achievement is 
zero. (Appendix C reports on the two studies that were 
reviewed but did not meet the CSRQ Center’s standards.) 

Evidence of Positive Effects for Diverse Student 
Populations 

Rating: @ 

Because no studies of Leona met the CSRQ Center’s 
standards, the impact of this model on student 



achievement for diverse student populations is 
unknown. Therefore, the rating in this subcategory 
is no rating. 

Evidence of Positive Effects in Subject Areas 
Rating: ® 

Because no studies of Leona met the CSRQ Center’s 
standards to review, the rating in this subcategory is 
no rating. 

E vidence of Positive Effects on 
Additional Outcomes 

Rating: 

Because no studies of Leona met the CSRQ Center’s 
standards, it was not able to evaluate the effects of 
Leona on additional outcomes. Therefore, the rating 
in this category is no rating. 



E vidence of Positive Effects on Parent, 
Eamily, and Community Involvement 

Rating: (@ 

Because no studies that met the CSRQ Center’s stan- 
dards examined the effects of Leona on parent, family, 
or community involvement, the rating in this category 
is no rating. 



E vidence of Link Between Research and 
the Model’s Design 

Rating: Q 

Based on documentation provided by the model, 
explicit citations support the inclusion component of 
the model. However, explicit citations for the following 
core components were not provided: organization and 
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governance, professional development, technical assis- 
tance, and technology. Therefore, the model rating for 
evidence of link between research and the model’s 
design is limited. 



E vidence of Services and Support to 
Schools to Enable Successful Implementation 

Evidence of Readiness for Successful Implementation 
Rating: (J 

Based on documentation provided hy the model, Leona 
offers a formal process to help school staff establish an 
initial understanding of the model.^ The model also 
offers a formal process for allocating school resources, 
such as materials, staffing, and time. However, the 
model provides only informal benchmarks for imple- 
mentation in the form of a management contract and 
an action plan. The contract and action plan do not 
include indicators of successful implementation. 
Therefore, according to the CSRQ Center’s standards, 
the rating for evidence of readiness for successful 
implementation is moderate. 

Evidence of Professional Development/Technical 
Assistance for Successful Implementation 

Rating: 9 

Leona provides ongoing training opportunities, such 
as workshops, peer coaching, capacity building, and 
sessions for new staff Additionally, the model pro- 
vides supporting materials for professional develop- 
ment that address all of Leona’s core components. 
However, the model offers only an informal plan to 
help build school capacity to provide professional 
development. Therefore, according to the CSRQ Center’s 
standards, the rating for evidence of professional 



development/technical assistance for successful imple- 
mentation is moderately strong. 



CZy entral Components 



Administrative Services 

Leona provides comprehensive management and 
start-up services to schools. Because Leona owns the 
contract for charter schools in Arizona, the company 
provides all start-up and management services to 
these schools. In the Midwest, Leona is hired by the 
charter school’s board of directors; therefore, the board 
may hire Leona to provide a limited number of man- 
agement services. The company has three categories 
for classifying these services: start-up, operations, 
and professional. 

■ Start-up. During the start-up phase, Leona provides 
consultation on locating and acquiring a school 
facility. Leona’s staff works directly with school 
boards to negotiate leases and purchases. In addi- 
tion, Leona offers such financial management as 
arranging short-term loans, restructuring debt, 
and providing grant support. 

■ Operations. According to Leona, its staff seeks to 
keep each school in compliance with local, state, 
and federal regulations and the school’s charter 
contract. To this end, Leona prepares timely reports 
for local, state, and federal governing bodies. Leona- 
managed schools also receive budget oversight, 
financial reporting, human resources, and market- 
ing support from Leona. Although the board of 
directors in the Midwest retains ultimate authority 
over the budget, Leona prepares the school budget; 
makes loan arrangements; oversees purchasing; 
coordinates an independent audit; and prepares 
budget reports for local, state, and federal 



'As noted in “Methodology,” the following subcategory, which has been used to rate school reform models in previous reports from the CSRQ Center, is not 
applicable to ESPs: Provider ensures initial commitment from schools. 
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governing bodies. Notably, Leona has a full-service 
grants department that helps all Leona-managed 
schools research, write, and track grant applications 
that earn schools additional funds for curricular 
programs, supplemental educational services, and 
technology 

■ Professional. Leona is responsible for recruiting, 
training, and tracking the certification of all school 
staff According to Leona, salaries and most benefits 
(e.g., medical, dental, and vision care and retirement 
savings programs) are paid in full by the company 
In addition, Leona performs an annual review of 
school staff and provides merit pay based on this 
review. Finally, Leona works collaboratively with 
the school leader (the ESP’s preferred term for 
principal) to develop a strategic plan for recruiting 
students and maintaining student enrollment. 
Leona-managed schools welcome all students; 
however, if the school reaches capacity, then student 
selection occurs through a lottery. 

Organization and Governance 

At the state level, each Leona-managed school in the 
Midwest is accountable to its charter authorizer and 
the school’s board of directors. In Arizona, Leona- 
managed schools are accountable directly to the charter 
authorizer. All Leona-managed schools must comply 
with legislation of the state charter. To this end, Leona 
works with key stakeholders (school boards, charter 
authorizers, and school staff) to maintain compliance 
and produce appropriate fiscal and evaluative reports 
on each Leona-managed school. 

At each school, Leona hires and trains the school 
leader. Because this position entails administrative 
and instructional leadership, Leona believes the term 
school leader is more appropriate than school principal. 
The school leader works closely with the school’s 



board of directors and Leona to plan and coordinate 
professional development sessions and onsite meetings, 
to select and implement a curricular program, and to 
develop a strategic plan for maintaining school enroll- 
ment. The school leader also collaborates with the 
school improvement team — which involves other school 
staff, school board members, parents, and students — 
to make decisions about school operations, such as the 
curriculum, the daily schedule, and extracurricular 
activities. 

In addition, Leona hires a curriculum coach for each 
Leona-managed school. This coach assists the school 
leader by mentoring teachers, overseeing instructional 
practices and the curricular program, and providing 
onsite professional development and technical assistance. 

Curriculum and Instruction 

Leona does not require schools to adopt a certain cur- 
ricular program. Instead, Leona’s staff provides guid- 
ance to the school leader and school improvement 
teams on specific curricular programs and strategies 
for meeting the requirements of the No Child Left 
Behind Act of 2001. Leona hopes that the selection of 
a curricular program will be a joint decision made by 
aU key stakeholders, including school staff, board 
members, Leona’s staff, and parents. Notably, current 
Leona-managed schools have selected such curricular 
programs as Direct Instruction, Success for All, and 
Core Knowledge.^ 

In addition, Leona hires a curriculum coach for each 
Leona-managed school. The coach’s major responsibility 
is to improve the instructional practices of Leona’s 
staff through professional development opportunities. 
The curriculum coach performs such duties as helping 
teachers align the curriculum with state and district 
standards, implementing the curriculum, writing lesson 
plans, and developing curriculum maps. The coach 



^The CSRQ Center reviewed Direct Instruction, Success for All, and Core Knowledge in a previous report: CSRQ Center Report on Elementary School 
Comprehensive School Reform Models. The report can be accessed at http://www.csrq.org. 
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also observes teachers and provides feedback to improve 
their instructional practices. 

The curriculum coach encourages teachers to adopt 
instructional strategies that meet the needs of students 
with multilevel abilities. Because Leona requires each 
student, including those with special needs, to be edu- 
cated in the general education classroom, teachers are 
responsible for adopting instructional strategies that 
meet the social and cognitive needs of each student. 
Special and general education teachers work collabo- 
ratively in the general education classroom to develop 
individualized instructional programs for all students. 
Leona provides both general and special education 
teachers with training on co-planning and co-teaching. 

To individualize instruction for each student, Leona 
recommends that teachers adopt the following instruc- 
tional strategies: cooperative learning, project learning, 
differentiated instruction, and learning centers. School 
staff at Leona-managed schools receives on- and off- 
site professional development on these instructional 
strategies. 

Scheduling and Grouping 

Leona works closely with the school’s board of direc- 
tors, school improvement teams, and the school leader 
to establish a schedule that meets the needs of the 
community. vMthough Leona does not require schools 
to adopt a specific schedule, the ESP does make rec- 
ommendations about scheduling practices for 
elementary/middle schools and high schools. In Leona- 
managed elementary/middle schools, it recommends 
a full-day kindergarten, an extended school day and 
year, and before and after school childcare. In Leona- 
managed high schools, the ESP recommends block 
scheduling; a 4-day school week, with an additional day 
for focused tutoring and study; and evening classes. 

Leona does not have specific grouping requirements 
beyond its commitment to full inclusion. Leona expects 



general and special education teachers to work coUab- 
oratively in the general education classroom to teach 
aU students, including those with special needs. 

Technology 

Leona requires technology for both instructional and 
administrative use. Leona provides all schools with 
technology support, including network management, 
hardware and software purchases, and technical trou- 
bleshooting. Most Leona-managed schools have a 
mobile, wireless computer lab. Leona also works with 
the school leader and the school’s board of directors to 
get the most use out of technology with a minimal 
impact on the school’s budget. Therefore, Leona does 
not require a specific number of computers per pupil 
or specific computer packages for core content areas. 

Monitoring Student Progress and Performance 

In the Midwest, Leona-managed schools are required 
to use state-mandated assessments to measure the 
progress of students. In addition to the state-mandated 
assessments, Leona expects teachers to use multiple 
forms of qualitative and quantitative assessments, such 
as project-based learning. In accordance with Leona’s 
full inclusion philosophy, teachers are expected to 
modify the curriculum so that students can progress 
at their own pace. 

In Arizona, Leona-managed schools use state-mandated 
assessments and formal benchmark assessments that 
are developed by Leona. The benchmark assessments 
delineate student progress by state or district standard. 
Teachers are expected to use the results of these assess- 
ments to monitor student and school performance and 
to assess their instructional practices. Leona-managed 
schools in Arizona also adhere to the full inclusion 
philosophy; therefore, teachers use multiple forms of 
assessments, and students are not penalized for not 
performing at grade level. 
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Leona conducts an annual evaluation of its schools. 
The evaluation reviews the school environment, cur- 
riculum, instruction, assessment, and professional 
development. The results of the evaluation help all 
Leona-managed schools to establish annual goals for 
academic progress. These goals and achievement data 
are published annually by Leona in a school-specific 
report. The report also briefly describes the school’s 
educational philosophy, professional development 
schedule, student enrollment, extracurricular activities, 
and parent satisfaction. 

Family and Community Involvement 

Leona recommends that all Leona-managed schools 
involve parents in daily school activities through vol- 
unteerism, school improvement teams, and parent- 
teacher associations. Leona works with the school 
leader and the school’s board of directors to develop 
a strategic communications and marketing plan for 
parents and the community. School staff members 
are encouraged to involve parents and community 
members in field trips, school tutoring, sports, clubs, 
and decision-making teams. Likewise, teachers are 
expected to encourage students to volunteer in their 
community. 

Leona assesses parent satisfaction annually through 
parent surveys. The results of these surveys and infor- 
mation on parent and community interaction with 
the school are published by Leona in the school’s 
annual report. 

Professional Development and Technical Assistance 

During the 2003-2004 school year, Leona began 
implementing a Quality Schools Initiative. Through 
this initiative, Leona seeks to provide a formal profes- 
sional development and technical assistance plan for 



aU Leona-managed schools. To this end, each Leona- 
managed school has a curriculum coach who oversees 
the school’s curricular and instructional program and 
provides onsite professional development and technical 
assistance to Leona’s staff. In addition, a team of 
employees from Leona coordinates monthly meetings 
with curriculum coaches and bimonthly meetings with 
school leaders. During these meetings and weekly 
school visits, a team of staff from Leona, who are 
dedicated to an individual school, monitors the school’s 
implementation progress and provides strategies for 
improvement. 

Leona also provides on- and offsite training on technol- 
ogy, inclusive practices. Sheltered English^ instruction, 
assessment, leadership, and instructional practices. 
Additionally, Leona works with the school leader and 
the school improvement team to identify topical areas 
in which teachers need training. When these areas are 
identified, Leona’s staff or contracted staff provides 
training on these topics. 

Implementation Expectations/Benchmarks 

Leona does not provide schools with implementation 
benchmarks or specific timelines for implementation. 
However, Leona does work with the school board to 
develop a management contract. The contract includes 
implementation tasks for the start-up process, finan- 
cial management, and the educational program and 
specifies who is responsible for completing each task. 
Moreover, Leona’s staff develops an action plan that 
includes an implementation checklist for each school. 

Special Considerations 

Leona seeks to provide a flexible educational program 
that meets the needs of the local community. Thus, 
the model does not require a specific curriculum or 



^Sheltered English instruction uses English as a Second Language instructional strategies in content area classes to support students in the acquisition of 
English and content area knowledge. 
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school calendar. One charter authorizer contacted hy 
the CSRQ Center noted that because the educational 
program is flexible, the types of services that are deliv- 
ered to each school vary. 

Leona-managed schools are also encouraged to estab- 
lish a strong link between home and school through 
after school childcare; parent volunteerism; and such 
extracurricular programs as clubs, sports, and interest 
groups. According to Leona, parent satisfaction is 
critical to a school’s success. 
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Mosaica Education 


Overview: 


Basic Model Information and Review Results 


Model Name: 


Mosaica Education 



Model Mission/Focus: Mosaica Education is based on eight pillars that form the basic tenets of the model: 



student achievement, professional development, integrated technology, parental 
involvement. Paragon curriculum, community support, secure environment, and 
extended learning time. Mosaica’s goal is for the pillars to work together to create 
superior academic and cost-effective results in each of its schools. 



Year Introduced in Schools: 


1997 








Grade Levels Served: 


K-8 








Number of Schools 

Total: 

40 


Urban: 

N/A 


Suburban: 

N/A 


Rural: 

N/A 




Costs 


Total Operating 
Costs: 


Training: 


Materials: 


Personnel: 


Other: 


Yearl 


Varies 


N/A 


N/A 


N/A 


N/A 


Year 2 


Varies 


N/A 


N/A 


N/A 


N/A 


Year 3 


Varies 


N/A 


N/A 


N/A 


N/A 


Years 4+ 


Varies 


N/A 


N/A 


N/A 


N/A 



1 . Evidence of Positive Effects on Student Achievement: 

a. Overall effects 0 

b. Evidence of positive effects for diverse student populations ® 

c. Evidence of positive effects in subject areas: 

Reading 0 

Math 0 

2. Evidence of Positive Effects on Additional Outcomes ® 

3. Evidence of Positive Effects on Parent, Family, and Community Involvement ® 

4. Evidence of Link Between Research and the Model’s Design ® 

5. Evidence of Services and Support to Schools to Enable Successful Implementation: 

a. Evidence of readiness for successful implementation ® 

b. Evidence of professional development/technical assistance for successful implementation ® 



^ = Very Strong ^ = Moderately Strong 3 = Moderate (J = Limited 0 = Zero 0 = Negative @ = No Rating 



This description is based on publicly available information, including the model’s Web site, regarding the model and its 
costs. The Comprehensive School Reform Quality Center attempted to obtain specific information, but this was not always 
possible. Areas in which exact information was not provided are marked by “N/A.” 



MOSAICA EDUCATION 



M odel Description 

The information presented in this description of Mosaica 
Education was collected using (a) Mosaica’s Web site 
and (b) responses from conversations with three charter 
authorizers who oversee the charter authorization 
process for schools using the model. Although contact 
was initiated with a representative of Mosaica, the 
Comprehensive School Reform Quality (CSRQ) Center 
was unable to conduct a conversation with the model 
provider. For more specific information on Mosaica, 
sites should directly contact the model. 

Mosaica Education is an education service provider 
(ESP) that was founded in 1997 by Dawn and Gene 
Eidelmam. Four years later, the model acquired 
Advantage Schools, Inc. and merged the models. 
According to the ESP’s Web site, Mosaica currently 
operates more than 40 elementary and middle schools in 
eight states. Mosaica generally partners with community 
groups, universities, or school districts to create new 
schools or to reform existing ones. Mosaica’s schools 
feature strong mentor and volunteer programs, an inter- 
disciplinary curriculum, and an extensive focus on tech- 
nology. The Paragon curriculum (developed by Mosaica 
and described in “Curriculum and Instruction”) is used 
in all schools and is based on Dr. Howard Gardner’s 
theory of multiple intelligences (Gardner, 1983). 

According to the CSRQ Center’s standards, the follow- 
ing were identified as core components of Mosaica: 
organization and governance; professional development; 
curriculum; instruction; technology; time and sched- 
uling; and parent, family, and community involvement. 
Core components are considered essential to successful 
implementation of the model. 

Mission/Focus 

According to its Web site, Mosaica’s mission is to open 
portals of opportunity for children and adults through 



excellence in education. The ESP claims that its schools 
serve as community pillars of life-long learning. 

Goals/Rationale 

According to its Web site, Mosaica’s philosophy of 
education is centered on the needs of each student. 
The model attempts to identify and address individual 
strengths through the use of the Paragon curriculum 
and Personalized Learning Plans (PLPs). Mosaica 
claims that its model is flexible and can be customized 
to meet the specific needs of a school or district that 
chooses to partner with the ESP. 

Eight pillars form the basic tenets of Mosaica: student 
achievement, professional development, integrated 
technology, parent involvement. Paragon curriculum, 
community support, secure environment, and extended 
learning time. The ESP believes that these pillars work 
together to provide learning opportunities for all stu- 
dents regardless of their specific learning needs. Strict 
requirements for schools in each of these areas are 
discussed in “Central Components.” 



osts 

The model’s costs vary depending on per-pupil spend- 
ing in a specific district. For more information on the 
costs of training, materials, and personnel, sites should 
directly contact the model provider. 



E vidence of Positive Effects on Student 
Achievement 

Evidence of Positive Overall Effects 
Rating: 0 

The CSRQ Center reviewed three quantitative studies 
that examined the effects of Mosaica on student 
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achievement. One of these studies met the CSRQ 
Center’s standards for rigor of research design. Upon 
review, the CSRQ Center considered the findings of 
this study to he conclusive, meaning the CSRQ Center 
has confidence in the results reported. This study used 
a matched-comparison group, quasi-experimental 
design. This study focused on fifth and eighth graders 
in one Mosaica school in Delaware that predominantly 
served African-American students from low-income 
families.^ The study reported findings from four 
separate achievement tests and found no statistically 
significant positive impact. Qn three of the achieve- 
ment measures, Mosaica showed no effect, and on one 
achievement measure Mosaica showed negative effects. 
Therefore, the rating for positive overall effects is zero. 
(Appendix D reports on the other two studies that 
were reviewed hut did not meet the CSRQ Center’s 
standards.) 

Evidence of Positive Effects for Diverse Student 
Populations 

Rating: ® 

The study that met the CSRQ Center’s standards and 
is considered to he conclusive did not examine the 
impact of Mosaica on student achievement for diverse 
student populations. Therefore, the rating in this suh- 
category is no rating. 

Evidence of Positive Effects in Subject Areas: Reading 
Rating: 0 

Qne study that met the CSRQ Center’s standards and 
was considered to he conclusive examined fifth- and 
eighth-grade students’ reading achievement data from 
the Delaware Student Testing Program. The study did 
not find statistically significant differences in favor of 
Mosaica’s students. The rating for reading is zero. 



Evidence of Positive Effects in Subject Areas: Math 
Rating: 0 

Qne study that met the CSRQ Center’s standards and 
was considered to he conclusive examined fifth- and 
eighth-grade students’ math achievement data from 
the Delaware Student Testing Program. The study did 
not find statistically significant differences in favor of 
Mosaica’s students. The rating for math is zero. 



E vidence of Positive Effects on 
Additional Outcomes 

Rating: (@ 

No studies were eligible for review that examined the 
impact of Mosaica on additional student outcomes. 
Therefore, the rating in this category is no rating. 

E vidence of Positive Effects on Parent, 
Eamily, and Community Involvement 

Rating: 

No studies were eligible for review that examined the 
effects of Mosaica on parent, family, or community 
involvement. Therefore, the rating in this category is 
no rating. 



E vidence of Link Between Research and 
the Model’s Design 

Rating: 

The CSRQ Center did not conduct a conversation with 
the ESP, nor was it able to find publicly available infor- 
mation to allow it to rate this dimension. Consequently, 
the model received no rating for its link between 
research and the model’s design. However, readers may 



'This school has terminated its agreement with Mosaica Education after the time of the posttest. 
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wish to review the research on the effectiveness of 
Howard Gardner’s multiple intelligences approach that 
is used as a building block for Mosaica’s instructional 
approach (Gardner, 1983). 



E vidence of Services and Support to 
Schools to Enable Successful Implementation 

Evidence of Readiness for Successful Implementation 
Rating: ® 

The GSRQ Genter did not conduct a conversation 
with the ESP and therefore, was not able to collect the 
evidence necessary to rate this category. Therefore, the 
model received no rating for evidence of readiness for 
successful implementation. 

Evidence of Professional Development/Technical 
Assistance for Successful Implementation 

Rating: ® 

The GSRQ Genter did not conduct a conversation with 
the ESP, nor was it able to find publicly available infor- 
mation to allow it to rate this dimension. Gonsequently, 
the model received no rating for evidence of profes- 
sional development/technical assistance for successful 
implementation. 



CZy entral Components 



Administrative Services 

Mosaica offers a wide range of administrative services 
to its schools through Spectra Services, a division of 
the company that focuses on the management of public 
and private schools. Services offered by the model 
include operations management (e.g., marketing and 
governance), financial management (e.g., payroll 
management and grant writing), facilities management 



(e.g., identifying school sites and renovating facilities), 
and human resources management (e.g., hiring and 
training personnel). In addition. Spectra Services can 
also provide technology and assessment services for 
both special education and general education students 
and supplemental education services as required 
under the guidelines of the No Ghild Left Behind Act 
of 2001. Administrative services are available to all 
schools regardless of the grade levels served. 

Mosaica works with both new and existing schools. Eor 
a new school, Mosaica begins by conducting a popula- 
tion analysis and meets with community members to 
determine the need for and interest in a charter school. 
The start-up process for an existing school begins with 
a charter school’s board of trustees that contracts with 
Mosaica to provide both educational and administrative 
services to a school. Mosaica prefers to partner with 
school districts but this is not always the case in prac- 
tice. Mosaica schools are open to any student who is 
qualified under state law for admission to the local 
public school system. 

Organization and Governance 

Mosaica does not require its schools to make any spe- 
cific changes to their governance structure or to hire 
any additional staff members. However, the model 
does require its schools to make some smaller organi- 
zational changes. Eor example, all Mosaica schools are 
required to have dedicated instructional blocks and to 
extend the school day and school year to increase the 
learning time available for students. Mosaica’s schools 
also use the model’s behavioral program and discipli- 
nary code of conduct. 

Curriculum and Instruction 

Mosaica requires all of its schools to use Paragon, an 
integrated and student-centered humanities and social 
studies curriculum that was developed by the model. 
Paragon is an interdisciplinary curriculum that is 
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organized around a series of essential questions. 
Although it is primarily a social studies curriculum, 
it also integrates music, technology, and the arts into 
the learning process. 

Paragon is based on the theory of multiple intelligences 
and is designed to address individual learning styles. 
The model believes that the interdisciplinary nature 
of Paragon, which uses music and art instruction in 
addition to the core subject areas, allows teachers to 
recognize and cultivate the success of each student in 
the most appropriate manner. The model further 
believes that Paragon can help students identify indi- 
vidual strengths that can in turn be used to support 
the learning process. To this end, the ESP requires 
teachers to use what it considers to be research-based 
and effective teaching strategies, such as journals, 
Socratic discussions, role plays, and graphic organizers. 

The curriculum fulfills all state and national history, 
social studies, and geography standards. Paragon is 
designed to balance cultural literacy with extensive 
hands-on learning opportunities while providing a 
chronological approach to the study of history. The cur- 
riculum presents histories of different countries through 
a series of cycles that repeat themselves over time. 
Students develop a large picture of history and associ- 
ated interrelationships by learning how ideas originate 
simultaneously in different areas. The curriculum focus- 
es more on fostering students’ understanding of sequen- 
tial events than on rote memorization of dates and facts. 

Daily lessons are organized around a series of essential 
questions that were developed by the model. The 
essential questions encourage students to connect the 
topics discussed with other topics and subject areas as 
well as with their life experiences with the hope of 
making the lessons seem more relevant and purposeful. 
Paragon aims to help students see all subjects as relevant 
parts of their lives rather than as separate disciplines. 

Paragon also integrates music and arts instruction 
into the curriculum. For example. Paragon integrates 



relevant music into each unit of instruction and focuses 
on how music and art in one era can influence develop- 
ments in another. Students are encouraged to approach 
each era of the curriculum through several angles, 
including singing, dancing, and performing. Paragon 
also uses arts to help enhance the communication 
skills of its students. The model believes that arts 
instruction enhances traditional learning instruction 
through a deeper level of analysis and self-expression 
than can be offered through a textbook. The model 
further believes that the use of such alternative teach- 
ing methods can help draw students who have been 
marginally successful into the learning process. 

Each Mosaica school follows a similar daily schedule. 
Mornings are dedicated to the core subject areas of 
reading, writing, math, and science. Afternoons are 
dedicated to Spanish, music, and physical education 
instruction in addition to a daily, 90-minute block that 
is dedicated to Paragon. In addition to the core subject 
areas and Paragon, students also learn about character, 
ethics, empathy, and self-esteem by studying great 
thinkers of the past and present. Mosaica introduces 
all students to foreign language instruction beginning 
in kindergarten. Mosaica also provides tutorial assis- 
tance to all struggling students through additional 
work with a classroom teacher, tutor, or classmate. 

Scheduling and Grouping 

Mosaica requires its schools to extend the school day 
and the school year to allow for additional learning 
opportunities. At most of Mosaica’s schools, the school 
day is extended by 1 hour and the school year by 20 
days compared with the traditional district schedule. 
The model believes that these changes allow students 
to master topics rather than to memorize facts. 

Mosaica does not require the use of any specific group- 
ing strategies within a school. However, the model 
does require schools to cap class size at 25 students. 
The model claims that the limited class size allows 



CENTRAL COMPONENTS 



66 



MOSAICA EDUCATION 



students to receive additional attention and to focus 
better on the subjects covered. 

Technology 

Mosaica places a strong emphasis on the use of tech- 
nology to enrich instruction and to aid with problem 
solving. The model requires all schools to use technol- 
ogy for instructional and noninstructional purposes. 
The model provides teachers and administrators with 
laptops and technology training sessions. The model 
also requires schools to secure enough computers to 
maintain a computer-to-student ratio of 1:3. All of 
Mosaica’s schools have an Intranet connection that is 
used to connect classroom computers to instructional 
and informational technology resources. 

Students use computers to research topics, express them- 
selves creatively, problem solve, interpret and organize 
data, and explore math and scientific principles through 
simulations. Mosaica believes that students need con- 
stant access to computers to gain true computer literacy. 
In addition to schoolwide computer labs, each class- 
room is equipped with computers for student use. 

Paragon provides several opportunities for hands-on 
computer work and Internet resources for further 
learning opportunities. Because Paragon frequently 
uses film clips from movies to engage students in 
history lessons, each classroom must have a TV and 
VCR. Overhead projectors are also provided in every 
classroom to support Paragon. 

Monitoring Student Progress and Performance 

Mosaica requires aU schools to develop and foUow PLPs 
for each student. PLPs are designed to help teachers 

(a) monitor a student’s attendance and behavior and 

(b) follow the academic growth of students both in 
the classroom and through the individualized tutorial 
programs that are available in reading and math. The 
model believes that such individualized student 



monitoring helps teachers to communicate better with 
students and parents. 

Students in all of Mosaica’s schools are tested using the 
Iowa Test of Basic Skills (ITBS), a norm-referenced 
standardized assessment. The model selected ITBS 
because it believes that the assessment is closely aligned 
with Paragon. 

For low-performing students, Mosaica uses an Integrated 
Learning System (ILS) to track student progress in 
math and language arts and to supplement classroom 
instruction. ISL aligns with state and national standard- 
ized tests, including the ITBS. ISL provides detailed 
reports that help teachers to evaluate the needs of each 
student. Teachers are encouraged to use the information 
when making instructional decisions. 

Family and Community Involvement 

All of Mosaica’s schools are required to establish rela- 
tionships with community and family members. Parents 
are expected to participate in one of the available service 
opportunities on a monthly basis. Examples of service 
opportunities include calling other parents to remind 
them about meetings, assisting with clerical work, and 
attending school programs. Mosaica also encourages its 
schools to establish partnerships with local businesses 
and often obtains financial support and direction from 
such businesses or from philanthropic foundations. 
Eight times a year, Mosaicas schools hold Paragon 
Night where individuals from families, universities, 
cultural centers, local businesses, and other community 
organizations are invited to the school to see student 
performances that focus on the historical era being 
studied through Paragon. 

Mosaica also encourages schools to develop a close 
connection between home and school to help enhance 
student performance. Mosaicas schools require parents 
to attend regular goal setting conferences to help 
determine the direction of a student’s PLP. Parents are 
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invited to take an active role in the decision-making 
process that determines their child’s PLP. Parents are 
also able to contribute to schoolwide decisions by 
serving on subcommittees or on the board of trustees. 
Each year, Mosaica’s schools conduct an annual parent 
satisfaction survey. 

Professional Development and Technical Assistance 

Mosaica assumes responsibility for hiring and training 
teachers and for providing ongoing professional devel- 
opment. All new staff members attend a 2-week sum- 
mer training session before the start of the school year. 
Mosaica also provides each school with 15-20 days of 
professional development each year. Topics and sched- 
ules vary depending on the terms of a school’s charter 
and the needs of the school. According to one charter 
authorize^ each school is assigned a regional vice 
president who visits the school on a weekly basis. For 
specific information on professional development and 
technical assistance provided by Mosaica, sites should 
directly contact the model provider. 

Implementation Expectations/Benchmarks 

Mosaica does not indicate that it provides schools 
with benchmarks or implementation expectations. For 
specific information on benchmarks and implementa- 
tion expectations, schools should directly contact the 
model provider. 

Special Considerations 

In conversations with three charter authorizers who 
have partnered with Mosaica, each felt that the model 
is beneficial, despite some challenges in implementation. 
Each of the charter authorizers praised Paragon and 
felt that its interdisciplinary nature and wide array of 
instructional strategies complement diverse student 
bodies. However, one charter authorizer cautioned 
that the service provider’s highly customizable nature 



spread staff thin while trying to meet the needs of each 
school’s unique situation. Another charter authorizer 
noted the potential for conflicts between the board of 
trustees and the service provider over the direction that 
a Mosaica school should take as it continues to grow. 
Overall, the three charter authorizers felt that the com- 
bination of administrative services and curriculum is 
an asset to the districts in which they work. 
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National Heritage Academies 



Overview: 


Basic Model Information and Review Results 


Model Name: 


National Heritage Academies (NHA) 


Model Mission/Focus: 


NHA contracts with authorizing agencies to create new public charter schools with the 
intent of helping all children achieve their greatest potential. 


Year Introduced in Schools: 


1995 


Grade Levels Served: 


K-8 



Number of Schools 

Total: Urban: Suburban: Rural: 

51 N/A N/A N/A 



Costs 


Total Operating 
Costs: 


Training: 


Materials: 


Personnel: 


Other: 


Yearl 


Varies 


N/A 


N/A 


N/A 


N/A 


Year 2 


Varies 


N/A 


N/A 


N/A 


N/A 


Year 3 


Varies 


N/A 


N/A 


N/A 


N/A 


Years 4+ 


Varies 


N/A 


N/A 


N/A 


N/A 



1 . Evidence of Positive Effects on Student Achievement: 

a. Overall effects 0 

b. Evidence of positive effects for diverse student populations ® 

c. Evidence of positive effects In subject areas ® 

2. Evidence of Positive Effects on Additional Outcomes ® 

3. Evidence of Positive Effects on Parent, Family, and Community Involvement ® 

4. Evidence of Link Between Research and the Model’s Design 0 

5. Evidence of Services and Support to Schools to Enable Successful Implementation: 

a. Evidence of readiness for successful implementation 3 

b. Evidence of professional development/technical assistance for successful implementation 3 



^ = Very Strong ^ = Moderately Strong 3 = Moderate (J = Limited 0 = Zero 0 = Negative @ = No Rating 



This description is based on publicly available information, including the model’s Web site, regarding the model and its 
costs. The Comprehensive School Reform Quality Center attempted to obtain specific information, but this was not always 
possible. Areas in which exact information was not provided are marked by “N/A.” 
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M odel Description 

National Heritage Academies (NHA) is an education 
service provider (ESP) that was founded in Michigan 
in 1995 by J. C. Huizenga. In the past decade, the model 
has been implemented by more than 50 schools in 
five states. 

According to NHA, it uses a back-to-basics approach 
to education that combines parental involvement, 
moral guidance, and rigorous academics. NHA 
chooses to establish new schools rather than take 
over existing ones. The model works with districts 
and local reform organizations to establish public 
charter schools in urban and suburban areas across 
the United States. 

According to the CSRQ Center’s standards, the follow- 
ing were identified as core components of NHA: 
organization and governance; curriculum; instruction; 
inclusion; technology; time and scheduling; instructional 
grouping; student assessment; data-based decision 
making; and parent, family, and community involve- 
ment. Core components are considered essential to 
successful implementation of the model. 

Model Mission/Focus 

According to the NHA, its mission is to challenge 
children to achieve their greatest potential. The model 
aspires to create a network of more than 200 charter 
schools that use high levels of parental involvement as 
their foundation. 

NHA combines academic excellence, parental involve- 
ment, moral guidance, measurable results, and highly 
qualified teachers to meet its commitment to all 
children. The model also strives to increase the quality 
of life and financial security of its employees and 
stakeholders. 



Four central pillars provide the foundation for all of 
NHAs activities and programs: 

■ Challenging academic programs. NHA uses a 
clearly defined curriculum and series of high 
standards to encourage academic progress in the 
core subjects of instruction. 

■ Teaching virtues. NHA teachers strive to help 
students develop strong moral values. 

■ Parent-teacher-child partnerships. NHA schools 
actively encourage parental involvement in all 
components of the program, both in and out of 
the classroom. 

■ Accountability. Each NHA school is held account- 
able to the state and district in which the school 
operates, the national center, and parents of all 
students who are enrolled at an NHA school. 

NHA believes that these four pillars work together 
to provide an effective learning environment for all 
students. 

Goals/Rationale 

According to its Web site, NHAs educational philoso- 
phy is grounded in the principles of Effective School 
Research. Developed by Professor Ronald R. Edmonds, 
Effective Schools Research defines an effective school 
as an institution in which all children can obtain the 
essential knowledge and skills needed to be successful 
in life (Edmonds, 1981). (Mr. Edmonds is the former 
director of the Center for Urban Studies at Harvard 
University.) Effective Schools Research recommends 
that schools follow a research-based program that can 
quantifiably improve student learning. NHA strives to 
create effective schools by integrating several unique, 
research-based characteristics that correlate with 
student achievement. 
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osts 



Evidence of Positive Effects in Subject Areas 
Rating: ® 



NHA schools receive funding from four sources: 

■ Approximately 90% of funding comes from one 
source — per pupil allocations from state tax dollars. 
This amount varies depending on per pupil spending 
in a specific district. 

■ Three other funding sources include local school 
districts, federal grants, and private investments. 

For more information on the specific costs of training, 
materials, and personnel, sites should directly contact 
the ESP. 



E vidence of Positive Effects on Student 
Achievement 

Evidence of Positive Overall Effects 
Rating: 0 

The CSRQ Center reviewed three quantitative studies 
of NHA on student achievement. None of the studies 
had sufficient rigor to meet the CSRQ Center s stan- 
dards for review. Therefore, the overall rating of the 
evidence of positive overall effects of this model on 
student achievement is zero. (Appendix E reports on 
the three studies that were reviewed hut did not meet 
the CSRQ Centers standards.) 

Evidence of Positive Effects for Diverse Student 
Populations 

Rating: ® 

Because no studies of NHA met the CSRQ Center’s 
standards for review, the impact of this model on stu- 
dent achievement for diverse student populations is 
unknown. Therefore, the rating in this suhcategory is 
no rating. 



Because no studies of NHA met the CSRQ Center’s 
standards for review, the rating in this suhcategory is 
no rating. 




vidence of Positive Effects on 



Additional Outcomes 



Rating: 



Because no studies of NHA met the CSRQ Center’s 
standards for review, it was not able to evaluate the 
effects of NHA on additional outcomes. Therefore, the 
rating in this category is no rating. 




vidence of Positive Effects on Parent, 



Eamily, and Community Involvement 



Rating: 



No studies that met the CSRQ Center’s standards 
examined the effects of NHA on parent, family, and 
community involvement. Therefore, the rating in this 
category is no rating. 



E vidence of Link Between Research and 
the Model’s Design 

Rating: (2) 

Based on documentation provided by the ESP, the 
model’s underlying theory is influenced by Edmonds’s 
principles of Effective Schools Research. (Eor more 
information, see the discussion of Effective Schools 
Research in “Goals/Rationale.”) However, NHA does 
not provide explicit citations for Edmonds’s principles. 
Therefore, based on the CSRQ Center’s standards for 
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review, the rating for evidence of link between 
research and the model’s design is zero. 



E vidence of Services and Support to 
Schools to Enable Successful Implementation 

Evidence of Readiness for Successful Implementation 
Rating: 3 

Based on documentation provided by the ESP, the 
model offers a formal process for allocating such school 
resources as materials, staffing, and time. However, 
the model does not provide formal benchmarks for 
implementation. Instead, NHA monitors the progress 
of its schools by enforcing strict performance standards 
and collecting monthly data on student performance. 
Therefore, according to the CSRQ Center’s standards 
for review, the rating for evidence of readiness for 
successful implementation is moderate.^ 

Evidence of Professional Development/Technical 
Assistance for Successful Implementation 

Rating: 3 

The model provides such ongoing training opportuni- 
ties as workshops, coaching, ongoing training, and 
sessions for new staff. Additionally, the model provides 
supporting materials for professional development 
that address all of its core components. However, the 
model offers only an informal plan to help build 
school capacity to provide professional development. 
Therefore, according to the CSRQ Center’s standards 
for review, the rating for evidence of professional 
development/technical assistance for successful imple- 
mentation is moderately strong. 



entral Components 



Administrative Services 

NHA considers itself to be a full-service model and 
provides a wide array of services to its schools. 
Administrative services, such as staff training and 
technology procurement, are provided to all schools 
regardless of the grade levels served. NHA also offers 
additional services, including 

■ Operations management, such as student recruit- 
ment and charter negotiations; 

■ Financial management, such as budget oversight 
and payroll management; 

■ Facilities management, such as securing school 
sites and renovating facilities; and 

■ Human resources management, such as hiring 
personnel and developing benefit packages. 

NHA does not work with existing schools. Instead, 
NHA opens and maintains only its own system of 
public charter schools. Each charter school is managed 
as an individual business unit, with the school princi- 
pal being held accountable for both educational and 
financial results. Generally, as the first step of the 
start-up process, NHA coordinates with a local school 
reform group in the community where it intends to 
open a new school. The ESP collaborates with the 
organization to write the charter and submit it to the 
relevant authorizing agency. After the charter is author- 
ized, NHA takes the lead in selecting and securing the 
school site and developing community support. The 
ESP conducts extensive demographic research before 
selecting the final site. NHA estimates that the entire 
start-up process takes between 3 months and 1 year. 



'As noted in “Methodology,” the following subcategory, which has been used to rate school reform models in previous reports from the CSRQ Center, is not 
applicable to ESPs: Provider ensures initial commitment from schools. 
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NHA is responsible for recruiting and admitting stu- 
dents to new NHA schools. Although NHA schools 
are open to all students, the number of applicants 
often exceeds the number of available spaces in a 
particular school. During an open enrollment period, 
interested students fill out applications to a school. 

If there are more applications than spaces, then a ran- 
dom-selection lottery is used to determine which stu- 
dents are admitted. After the open enrollment period 
ends, non-admitted applicants are placed on a waiting 
list, in case additional spaces become open. NHA 
gives preference to siblings of previously admitted 
students. 

Organization and Governance 

Each school is held responsible to several groups of 
individuals at different levels. In addition to the state 
board of education and NHA itself, each NHA school is 
overseen by both the chartering agency that authorized 
the initial charter and an academy board of directors 
that is established at each school. The board of directors 
comprises five to seven community volunteers. The 
primary responsibility of the board is to ensure that the 
school complies with all relevant laws and regulations. 

Each NHA school is governed by both the principal 
and the school leadership team (SET). The ESP believes 
that the principal is instrumental in shaping the overall 
success of each school and expects him/her to serve as 
both the instructional leader and the general manager 
of the school. The principal also serves as a member 
of the SET Specific duties of the principal, as outlined 
by NHA, include hiring and mentoring teachers, 
attending national conferences and professional devel- 
opment activities, granting release time for teachers to 
attend professional development activities, promoting 
parental involvement, approving the use of instruc- 
tional programs and materials, and scheduling common 
planning periods for teachers. 



The SET meets monthly under the leadership of the 
principal to discuss and review the school’s progress. 
The SET is responsible for developing the school 
improvement plan for subsequent years. The composi- 
tion of the team varies from school to school but 
generally consists of the school principal and the 
chairpersons from each parent committee. 

Each teacher at an NHA school is expected to hold a 
valid teaching certificate. NHA hires teachers through 
a selective screening process that includes parental 
input. Teacher pay is based on merit rather than sen- 
iority. NHA teachers work in a team setting to promote 
a dynamic learning environment for all students and 
use both direct and indirect instructional strategies to 
teach a multimethod curriculum using research-based 
best practices. NHA schools limit class size to 19-25 
students in kindergarten and 24-28 students in 
grades 1-8. 

All NHA schools follow a strict disciplinary code that 
encourages personal responsibility and universal 
respect. The ESP believes that by promoting good 
behavior, schools are also promoting student achieve- 
ment. Thus, NHA has developed a Student Code of 
Conduct and a disciplinary system of six levels that 
range from verbal warning to expulsion. NHA also 
requires students in all grades to follow a strict 
dress code. 

NHA can be implemented in both urban and suburban 
settings. All NHA schools begin as K-5 schools during 
the 1st year. Generally, one grade is added to the school 
each year of implementation through grade 8. A typical 
NHA school begins with 15-17 classrooms of grades 
K-5 and ends with 26-28 classrooms of grades K-8. 
This growth model increases a school size from 
roughly 360 to 650 students over a 4-year period. The 
model provides the same services to students in all 
grade levels. Currently, NHA does not provide services 
to high schools. 
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Curriculum and Instruction 

NHA focuses on providing students with a liberal arts 
education that provides a solid foundation in the core 
subjects of math, English, history, geography, govern- 
ment, economics, science, art, music, and physical 
education. NHA requires schools to use a combination 
of published curricula and model-developed curricula 
in all grade levels and recommends that schools spend 
twice as much time on language arts and math instruc- 
tion as on other core subject areas. 

For reading and writing instruction, NHA requires 
school to use the Open Court curriculum 
(http://www.sraonline.com). Through Open Court, all 
students, particularly those in grades K-2, receive 
intensive phonics instruction. Comprehension and 
fluency are also emphasized, taught, and evaluated at all 
grade levels. Saxon Math is used as the core math cur- 
riculum (http://saxonpublishers.harcourtachieve.com). 
The math program is based on the philosophy that 
students learn best through repetition of ideas. Sample 
lessons for both subject areas are provided by the 
developers of each curriculum. 

In other subject areas, the model combines the Core 
Knowledge sequence with model-developed and 
teacher-developed materials to provide instruction 
(http://www.coreknowledge.org).^ For example, NHA 
schools do not use textbooks for history or geography 
classes. Instead, teachers are expected to develop mate- 
rials that align with the Core Knowledge sequence. 

The Core Knowledge sequence accounts for more than 
half of the required curricula; the model-developed 
materials are designed to fill in instructional gaps. The 
NHA curriculum is aligned with Core Knowledge in 
the following subject areas: math, history, geography, 
government, science, English, and reading. 

In addition to the core curriculum requirements, NHA 
schools must also integrate moral values instruction 



into all subject areas. Each month, a different moral 
value is selected for instruction from a list based on 
the Greek Cardinal Virtues of justice, temperance, for- 
titude, and prudence. The moral values are rotated so 
that each one is selected for instruction only once every 
2 years. NHA students are also continually taught to 
look for role models and heroes. The ESP considers 
individual character development to be a cornerstone 
of its instructional philosophy. Character development 
is not treated as a separate curriculum but is integrated 
into all core subject areas as a constant theme. All stu- 
dents at NHA schools are expected to demonstrate 
certain moral values. 

All NHA teachers are instructed to create teacher- 
directed classrooms. Teachers are expected to serve 
as leaders and role models for all students. Specific 
duties include 

■ Preparing lesson plans that are based on previous 
knowledge of the subject, 

■ Determining the most effective way to present 
instructional materials, 

■ Engaging students in thought provoking lessons, 

■ Continually monitoring student comprehension 
throughout daily lessons, and 

■ Providing continued learning opportunities and 
practice with instructional materials. 

In teacher-directed classrooms, teachers are expected 
to direct instruction so that (a) all necessary objectives 
are met every day and (b) all students are able to master 
the content. 

Scheduling and Grouping 

The ESP schedules a 7-hour school day, with a mini- 
mum of 6 hours of instructional time. NHA does not 



^The CSRQ Center reviewed the Core Knowledge model in a previous report: CSRQ Center Report on Elementary School Comprehensive School Reform Models. 
The report can be accessed at http://www.csrq.org. 
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require its schools to follow a particular calendar. 
Instead, schools generally follow the calendar of the 
school districts in which they are located. NHA does 
require each of its school to have a minimum number 
of hours per day and days per year during which 
students must he in school. NHA schools are allowed 
more scheduling flexibility on a daily basis than on an 
annual one. The ESP encourages the use of dedicated 
instructional blocks but does not require it. 

The ESP claims that inclusion of students with disabil- 
ities in the general curriculum is a central component 
of the model but did not provide any information 
about specific inclusion strategies. 

Technology 

NHA takes the lead in planning, implementing, and 
supporting the use of technology in its schools and is 
responsible for leasing an adequate number of com- 
puters to each school. Computers, available for both 
instructional and noninstructional purposes, are used 
by both teachers and students. Each classroom at an 
NHA school has at least one computer that is connected 
to NHA’s computer network, and all schools have a 
media center with additional computers. NHA has also 
developed a series of grade-level computer proficiency 
goals to ensure that computers are used effectively in 
the classroom. Other uses of technology may include 
developing keyboarding skills, simulating a business 
plan, or analyzing data from a science experiment. 

All computers are also equipped with AtSchool soft- 
ware, which serves as the operating system for all 
NHA schools. AtSchool is an Internet-based program 
that includes math and reading/writing programs 
(for instructional purposes) and communications, 
accounting, and recordkeeping software (for nonin- 
structional purposes). Each school is required to use 
AtSchool. Eor more information on the costs of 
AtSchool, schools should directly contact the ESP. 



Monitoring Student Progress and Performance 

NHA schools are expected to follow the assessment 
schedules required by the state and district in which 
they are located. In addition, the ESP requires schools 
to administer the Northwest Evaluation Assessment 
Measures of Academic Progress (MAP) three times a 
year in grades 2-8. MAP is administered in the fall 
and spring and includes an optional mid-year test. 

This computer-based examination is administered to 
more than 3 million students nationally each year and 
is capable of providing instant results. Assessment 
data is used to group students across classes into three 
achievement levels. Student achievement data is also 
used to guide instructional strategies and to identify 
students in need of intervention. 

NHA does not have formal strategies in place to 
evaluate its effectiveness but does perform evaluations 
in some of its schools. Evaluations, using a variety of 
assessment tools, are generally conducted in low- 
performing schools. Both internal and external evalu- 
ators are used. NHA provides information to each of 
its schools that indicates where the particular school 
ranks among its peers. Implementation is monitored 
informally at the school level by surveys, observations, 
and student achievement data. The model expects 
each school to use implementation data to develop 
schoolwide goals for subsequent years. 

Family and Community Involvement 

NHA believes that all of its schools are held accountable 
to the students’ parents and strongly encourages a high 
level of participation from family and community 
members. Orientation meetings and parent-teacher 
conferences are held regularly. Parents are also asked 
to volunteer in the classroom or after school and to 
serve on the SET or academy board of directors. Each 
of these activities is designed to sustain family involve- 
ment in the school for the entire academic year. The 
AtSchool program allows parents to access — from their 
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homes — newsletters, teacher messages, and individual 
records of their children. 

NHA works with individual schools to establish parent 
committees at each school. The committees help to set 
the direction of a school, although the final responsi- 
bility for setting policy lies with the academy board of 
directors and the SLT. The committees deal with a 
wide range of areas, such as moral focus, building and 
grounds, library, booster, hospitality, and ambassadors. 
The ESP also works with each school to establish a 
parent room to help parents feel welcome in the school. 
The model strongly urges schools to develop and use 
parent/family contracts to ensure a continued level of 
involvement from family members. Each fall and spring, 
NHA schools conduct a Parent Satisfaction Survey to 
solicit feedback and input from family members. 

Professional Development and Technical Assistance 

NHA has developed an informal professional develop- 
ment plan that is available to all schools. Each new 
NHA school is required to use the model’s professional 
development plan during its 1st year of operation. 
After the 1st year, the principal decides whether to 
continue using the plan. Mentoring teams are available 
to provide onsite coaching and technical assistance 
as needed. 

Before a school opens, all new teachers attend a 2-week 
summer training session. NHA generally partners with 
local universities to offer the session in the format of 
an accredited summer school course. After the initial 
training session, NHA works with the school’s principal 
to determine the topics and delivery format of addi- 
tional training sessions. NHA claims that professional 
development and technical assistance materials are 
available for each of the core components listed under 
“Model Description.” 

Any school that is considered to be underperforming 
must send its staff members to a mandatory onsite 



intervention workshop. Apart from the intervention 
workshops and the initial training, NHA does not have 
mandatory professional development sessions for any 
subject area. All professional development providers 
are trained by NHA at its national headquarters in 
Michigan. 

Implementation Expectations/Benchmarks 

NHA has not developed specific implementation 
timelines and benchmarks. However, it has developed 
a set of annual implementation goals that schools are 
expected to work toward. The ESP has also set up 
procedures that allow schools to monitor and report 
student progress, monitor instructional methods, 
certify completion of academic objectives, and solicit 
and monitor parent satisfaction. NHA collects monthly 
student performance data that when analyzed, can be 
compared with past and current performance and 
against other schools. 

Special Considerations 

The model claims to take aU responsibility for ensuring 
a school’s success. If a school is not performing at its 
highest potential, NHA supplies additional resources 
at no extra cost. However, in a conversation with one 
charter authorize^ it was noted that NHA is not always 
able to correctly interpret the implementation data 
that it collects to identify potential problems. Another 
charter authorizer noted that after a period of staff 
turnover within the company, NHA has an increased 
emphasis on training regional consultants to provide 
assistance to schools on a regular basis to supplement 
the support provided by the national center. 

Ml three charter authorizers contacted by the CSRQ 
Center felt that even with some shortcomings, NHA 
provides valuable services to its schools. Each autho- 
rizer stressed the accessibility and competence of the 
ESP’s staff members. 
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SABIS Educational Systems, Inc. 



Overview: 



Basic Model Information and Review Results 



Model Name: 



SABIS Educational Systems, Inc. 



Model Mission/Focus: 



SABIS’s philosophy is built on the belief that all students can learn and all students 
should have access to a university education. SABIS strives to foster a love for learning 
and to create well-rounded citizens and leaders. 



Year Introduced in Schools: 


1995 (in the United States) 






Grade Levels Served: 


Pre-K-12 








Number of Schools 










Total: 




Urban: 


Suburban: 


Rural: 




sV 




N/A 


N/A 


N/A 




Costs 


Total Operating 
Costs: 


Training: 


Materials: 


Personnel: 


Other: 


Yearl 


N/A 


N/A 


N/A 


N/A 


N/A 


Year 2 


N/A 


N/A 


N/A 


N/A 


N/A 


Year 3 


N/A 


N/A 


N/A 


N/A 


N/A 


Years 4+ 


N/A 


N/A 


N/A 


N/A 


N/A 



1 . Evidence of Positive Effects on Student Achievement: 

a. Overall effects 

b. Evidence of positive effects for diverse student populations 

c. Evidence of positive effects in subject areas 

2. Evidence of Positive Effects on Additional Outcomes 

3. Evidence of Positive Effects on Parent, Family, and Community Involvement 

4. Evidence of Link Between Research and the Model’s Design 

5. Evidence of Services and Support to Schools to Enable Successful Implementation: 

a. Evidence of readiness for successful implementation 

b. Evidence of professional development/technical assistance for successful implementation 



(NRJ 

® 

® 

® 

® 

® 

(nrI 



) = Very Strong ^ = Moderately Strong 3 = Moderate = Limited 0 = Zero Q = Negative @ = No Rating 



This description is based on publicly available information, including the model’s Web site, regarding the model and its 
costs. The Comprehensive School Reform Quality Center attempted to obtain specific information, but this was not always 
possible. Areas in which exact information was not provided are marked by “N/A.” 



'SABIS operates 10 schools in the United States. 



SABIS EDUCATIONAL SYSTEMS, INC. 



M odel Description 

The information presented in this description of 
SABIS Educational Systems, Inc., was collected using 
(a) SABIS’s Web site, (b) materials sent to the 
Comprehensive School Reform Quality (CSRQ) Center 
by SABIS, and (c) responses from one conversation with 
a charter authorizer who oversees the charter authori- 
zation process for a school that uses SABIS. Although 
contact was initiated by the CSRQ Center, it was unable 
to conduct a conversation with a representative from 
SABIS and four other charter authorizers. 

SABIS Educational Systems, Inc., was incorporated in 
1985 in Minnesota. SABIS owns the rights to the 
SABIS Edge Education Program in North America, 
South America, and Europe. SABIS operates 3 1 schools 
in 11 countries. These schools are part of the SABIS 
School Network. The first SABIS charter school in 
the United States opened in 1995 in Springfield, 
Massachusetts. 

According to the CSRQ Centers standards, the following 
were identified as core components of SABIS: organi- 
zation and governance; professional development; 
technical assistance; curriculum; instruction; inclusion; 
technology; student assessment; data-based decision 
making; and parent, family, and community involve- 
ment. Core components are considered essential to 
successful implementation of the model. 

Mission/Focus 

According to SABIS, its philosophy is that aU students 
can learn, even those of average or slightly below 
average ability. SABIS believes that a core curriculum 
in conjunction with a methodology of mastering key 
concepts is necessary to reach this end. In addition to 
academics, SABIS’s philosophy includes developing 
positive social skills and valuing and respecting cultural 
differences to become responsible citizens. 



Goals/Rationale 

SABIS’s Web site and materials indicate that the 
founders’ goal is for students to recognize the impor- 
tance of higher education, attend and be successful in 
college, be life-long learners, and be responsible citizens. 



C osts 

SABIS’s schools are public charter schools that are fund- 
ed by the public school system. Eor more information 
on the costs of training, materials, and personnel, 
schools or districts should contact SABIS directly. 



E vidence of Positive Effects on Student 
Achievement 

Evidence of Positive Overall Effects 
Rating: ® 

No quantitative studies that examined the effects of 
SABIS on student achievement were available for review 
by the CSRQ Center. Therefore, the overall rating in 
this subcategory is no rating. 

Evidence of Positive Effects for Diverse Student 
Populations 

Rating: ® 

Because no studies of SABIS were available for review by 
the CSRQ Center, the impact of this model on student 
achievement for diverse student populations is unknown. 
Therefore, the rating in this subcategory is no rating. 

Evidence of Positive Effects in Subject Areas 
Rating: ® 

Because no studies of SABIS were available for review, 
the rating in this subcategory is no rating. 
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JZ/ vidence of Positive Effects on 
Additional Outcomes 

Rating: @ 



Because no studies of SABIS were available for review 
by the CSRQ Center, the effects of SABIS on additional 
outcomes could not be evaluated. Therefore, the model 
rating is no rating. 




vidence of Positive Effects on Parent, 



Eamily, and Community Involvement 



Rating: @ 



evidence necessary to rate this dimension. According 
to the CSRQ Center’s standards, the model received 
no rating for evidence of readiness for successful 
implementation. 

Evidence of Professional Development/Technical 
Assistance for Successful Implementation 

Rating: ® 

The CSRQ Center did not conduct a conversation with 
SABIS, nor was the Center able to find publicly avail- 
able information to rate this dimension. Consequently, 
the model received no rating for evidence of profes- 
sional development/technical assistance for successful 
implementation. 



No studies that examined the effects of SABIS on parent, 
family, or community involvement were available for 
review by the CSRQ Center. Therefore, the model rating 
is no rating. 




entral Components 



Administrative Services 




vidence of Link Between Research and 



the Model’s Design 



Rating: @ 



The CSRQ Center did not conduct a conversation with 
SABIS, nor was the Center able to find publicly available 
information to rate this category. Consequently, the 
rating in this category is no rating. 



vidence of Services and Support to 



Schools to Enable Successful Implementation 



Evidence of Readiness for Successful Implementation 
Rating: ® 

The CSRQ Center did not conduct a conversation with 
SABIS. Therefore, the Center was not able to collect the 



As described on its Web site and in publicly available 
information and according to a charter authorizer who 
worked with the model, SABIS provides a wide array of 
administrative services to its schools. According to the 
charter authorizer, charter applications, and materials 
that SABIS provided to the CSRQ Center, the model 
provides financial management services, facilities man- 
agement services, and human resources management 
services. SABIS works with start-up schools to provide 
administrative services that are tailored to each school’s 
needs. These services are outlined in the contract 
agreement between SABIS and its clients. 

SABIS has established an admissions process for stu- 
dents who are interested in attending one of its schools. 
According to Web sites of SABIS’s schools, interested 
students must complete an application, submit recom- 
mendations, attend an informal meeting with the 
director of admissions, and take a placement test. 
SABIS has a nonselective admissions process. 
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Organization and Governance 

SABIS does not convert schools or take over and 
reconstitute existing schools. Instead, SABIS serves as 
managing partner with a school’s founders and works 
with the school’s hoard of directors to implement the 
model. The hoard of directors is in charge of the school, 
oversees the model’s progress, and submits charter 
applications and renewals to its state charter office. 

According to charter applications provided hy SABIS 
to the CSRQ Center, SABIS offers options for the degree 
of control that SABIS has in governing the school: 

■ In some cases, SABIS emphasizes site-hased control, 
whereby the principal and the board of directors run 
the school and set all policies. In this site-based 
control model, the board of directors oversees the 
school’s fiscal, organizational, and academic opera- 
tions and regularly convenes committees to meet 
with SABIS staff and school staff. 

■ In other cases, SABIS is contractually responsible 
for the day-to-day management and operation of 
the school. Under this complete management 
model, SABIS provides all recruitment, training, 
and supervision of staff. 

SABIS stresses the importance that all school staff 
commits themselves to the SABIS mission: preparing 
all students for higher education. Parents and students 
must also commit to the mission and play supportive 
roles through the Parent Connection and the Student 
Life Organization (SLO). 

Curriculum and Instruction 

Curriculum and instruction are the primary foci and 
building blocks of the SABIS educational system. The 
SABIS Edge Education Program encourages students 
to strive for excellence and develop confidence in 
themselves. The program has eight features: 



■ Screen aU new students for pre-existing learning gaps 

■ Monitor the progress of all students to identify 
ongoing learning gaps 

■ Provide intensive interventions to fill gaps that 
do exist 

■ Use a comprehensive curriculum to teach essential 
skills and knowledge required for academic success 

■ Implement instructional strategies to maximize 
student engagement and learning 

■ Use peer tutors 

■ Develop a student life program that focuses on 
building positive student attitudes 

■ Establish a safe and secure environment 

The range of products and services available varies 
depending on the management agreement between 
SABIS and the individual school. Under complete 
management, products and services for pre-K and 
K-12 include a complete curriculum that is aligned 
with state standards, more than 450 concept-targeted 
books, software systems, and a computerized testing 
and reporting system. 

SABIS emphasizes skills in the following areas: reading, 
writing, communicating, abstract reasoning, problem 
solving, and critical thinking. Knowledge of a second 
language is stressed and begins at the preschool level. 

According to SABIS, the SABIS Edge Education 
Program provides a structured curriculum that is 
aligned with state standards. SABIS’s schools can also 
use SABIS’s books that are coordinated with the cur- 
riculum. Curricula are available for English, Spanish, 
math, social studies, science, art, music, health, physical 
education, and computers. 

SABIS believes in a hierarchy of subjects and that not 
all subjects are of equal value. According to SABIS, a 
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subject’s importance is related to how central it is to a 
student’s future academic success. Math and English 
are deemed the two most important subjects. Foreign 
language instruction is considered the third most 
important subject. Because math is the precursor to 
science and English is the precursor to history, math 
and English take precedence over science and history. 

The math program teaches the four basic math opera- 
tions and also requires applying math to everyday life 
and understanding the language of math and the theory 
and logic behind it. Students are expected to be able to 
teach others what they have learned. Proficiency in 
English and a foreign language is considered vital. 
According to the model, solid knowledge of languages 
enhances a student’s ability to think logically, appreci- 
ate literature and other cultures, and communicate 
with others. 

Science is taught in an order different from most 
schools. Students take physics first, with biology and 
chemistry coming later. The math skills necessary for 
learning physics are taught in a pre-calculus class. 
Examples and problems are used to apply science to 
everyday life and to emphasize concepts. 

SABIS’s social studies content is drawn from geography, 
history, and political science. Students are encouraged 
to be culturally aware and to have an appreciation of 
other cultures. SABIS claims that this training creates 
responsible and active citizens. 

At the K-5 level, SABIS offers students a foundation of 
knowledge and skills to prepare them for more rigorous 
courses of study in the upper grades. Teaching methods 
of the SABIS Edge Education Program include memo- 
rization, phonics instruction in reading, and drills in 
math to build a foundation for more advanced learning. 

The SABIS Edge Education Program believes in whole 
class instruction using direct instruction. Students are 
assessed weekly to monitor progress and to identify 
gaps in learning from missed concepts. Students who 



need extra help to learn missed concepts or who are 
working below grade level are put in an intensive 
program with lower student-to-teacher ratios. Students 
in the intensive program are tested weekly and return 
to their regular classroom when their performance is 
back on target. 

The premise is that missed concepts and lack of pre- 
requisite knowledge, not a lack of ability, can result in 
poor academic performance or poor motivation. 
Preventing gaps and catching and closing possible gaps 
right away is essential for student academic success. 

Students are encouraged to participate actively in lessons 
and classroom discussions. Teachers use class time for 
academics and curriculum-related activities and expect 
students to be on-task. Teachers use pacing charts that 
are provided by SABIS to ensure that each class is 
doing the same concept and lesson on the same day. 

Efficiency is the goal of the SABIS Edge Education 
Program; teaching a set of knowledge or skills “in the 
shortest time possible with the least effort” avoids 
wasting time and increases time for additional learning 
or play. SABIS’s pacing charts are intended to help 
teachers maximize content coverage without giving 
up mastery of the material. The pacing charts tell the 
teachers what and how much to teach in any given 
class period. 

Teachers use a “point system” to implement maximum 
content coverage and pacing. The point system helps 
teachers focus on what they will teach and students 
focus on what they will learn. Teachers list concepts 
one point at a time, for example: 

■ Identifying the order of colors in the spectrum 

■ Ensuring gender agreement of articles and nouns 

Teachers introduce each point with a presentation, 
explanation, examples, and questions. They then lead 
an activity to apply the concept and check for under- 
standing. Teachers monitor student progress and 
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reteach as necessary. Student group leaders, or prefects, 
check the work of their peers. 

SABIS’s founders believe that efficient teaching includes 

■ Development of academic materials to fit student 
needs; 

■ Instructional strategies that encompass the 
appropriate activities and pacing to maintain 
momentum and interest; 

■ Selection of objectives at the appropriate level; 

■ Explicit instruction of the objectives; 

■ Application of basic principles of learning, such as 
maintaining motivation and encouraging retention 
and transfer; 

■ Instruction that is adjusted based on student 
progress; and 

■ Strong classroom management. 

SABIS goes a step further in defining efficient teaching. 
After identifying skills and concepts for a class, efficient 
teaching involves identifying which concepts and 
skills are essential and which ones are nonessential. 
Essential concepts are necessary for future learning; 
therefore, 100% mastery of essential concepts is 
required for grade advancement. In addition, some 
nonessential concepts are included in the curriculum, 
and some mastery above the minimum is required. 
Students must demonstrate mastery of concepts 
through testing to be promoted to the next level. 

Scheduling and Grouping 

SABIS groups students by ability and concepts 
mastered rather than by age. The model believes in 
whole class instruction. Within a class, students are 
divided into groups of four students. Each group has 
a student leader who can help other students in the 
group. When students are removed and put in the 



intensive program, they are tested periodically to 
determine when they can return to the regular class. 

SABIS offers an intensive English immersion program 
that emphasizes reading for students with limited English 
proficiency. This program promotes acquisition of 
English without diminishing the students first language. 

Technology 

SABIS offers the Academic Monitoring System (AMS), 
a computerized testing and reporting system to assess 
and track student achievement. AMS is proprietary to 
SABIS. Software for this system is available from 
SABIS under a management agreement. AMS is used 
to detect gaps in learning starting in fourth grade. 

Computers are used in all classrooms for lessons and 
for testing. Eor more information on the availability 
and costs of technology, schools or districts should 
contact SABIS directly. 

Monitoring Student Progress and Performance 

Monitoring student progress through regular assess- 
ments is an integral part of the SABIS model. Weekly 
assessments are meant to prevent learning gaps by 
helping teachers identify when “real time” remedial 
action is needed to catch students who may be falling 
behind. With some management arrangements, 
computerized tests, scoring, and feedback are available 
to assist teachers with tracking student achievement. 

Assessment takes place in these areas: oral language, 
vocabulary, spelling, grammar, reading comprehension, 
composition, and memorization. Eormal assessments 
begin in first grade, and by fourth grade, students are 
tested weekly. A comprehensive final exam is given at 
the end of each term. Report cards are distributed each 
term, and parents are informed via interim reports if 
their child is in danger of failing. In addition, teachers 
hold parent conferences twice a year. 
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SABIS uses AMS to test students weekly, identify learn- 
ing gaps, and provide immediate feedback to teachers. 
The computerized tracking ensures that SABIS has strict 
accountability for students, teachers, and administrators. 

Family and Community Involvement 

SABIS requires parents to participate in the Parent 
Connection, a parent-led organization. This program 
is designed to keep parents involved in their children’s 
learning process. It is also used for fundraising. 

Parents can get involved in such areas as new family 
orientation, an international fair, community service, 
and social events. 

Professional Development and Technical Assistance 

According to literature on SABIS and a charter autho- 
rizer, the model offers extensive training on its methods 
to principals and teachers before the school opens. 

A 1- to 2-week orientation program for new staff is 
provided during the summer (before the start of the 
school year). SABIS also provides a year-end review 
and follow-up training. 

Evaluators from SABIS’s national office spend time in 
schools to identify needs and devise improvement 
strategies for individual staff Throughout the year, 
formal and informal evaluations and recommendations 
on areas in need of improvement are made by admin- 
istrators, department heads, and peers who visit and 
conduct classroom observations. Teachers receive job- 
embedded professional development as they work in 
grade-level teams to observe each other’s instruction 
and to provide feedback and support. 

Implementation Expectations/Benchmarks 

The model does not provide formal benchmarks that 
apply across all SABIS schools. Rather, specific cur- 
riculum guidelines, pacing charts, and instructional 



guides form SABIS’s implementation expectations and 
benchmarks. In addition, each SABIS charter school 
has an accountability plan that outlines academic and 
organizational goals. The school’s charter application 
states specific measures as to how the school will 
achieve its goals. Here are two examples of perform- 
ance objectives: 

■ Ml students at the SABIS International Charter 
School will achieve mastery of computation and 
problem-solving skills necessary for proficiency 
in math. 

■ The administration of the SABIS International 
Charter School will ensure the safety of the school 
campus for all students and staff 

SABIS staff are in the schools (depending on the con- 
tract agreement) to monitor progress of and fidelity to 
the SABIS method. 

Special Considerations 

The SABIS model includes an SLO for all grade levels. 
This program emphasizes learning life skills through 
real-life experiences. The SLO encourages students to 
lead activities to learn organizational and leadership 
skills. The program also aims to help students to form 
friendships and become constructive members in their 
community. Students in an SLO organize peer tutoring, 
plan social events, organize community service projects, 
and produce a school newspaper. Students who partic- 
ipate in an SLO take an Honor Code Pledge that 
emphasizes respect for self and others. 

The Prefect System is part of the SLO. This program 
gives students varying degrees of leadership responsi- 
bility to assist their peers. Lor example, prefects may 
tutor peers, monitor hallways, or assist in the library 
or computer lab. 

The preceding description of the SABIS method was 
based on publicly available information and a 
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conversation with one charter authorizer. The CSRQ 
Center was unahle to conduct a conversation with 
SABIS. For more specific information on SABIS, schools 
or districts should contact the model provider directly. 
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White Hat Management (HOPE Academies) 



Overview: 



Basic Model Information and Review Results 



Model Name: 



White Hat Management (HOPE Academies)^ 



Model Mission/Focus: 



Year Introduced in Schools: 



HOPE Academies believe that there is hope for every child and help students reach 
their academic potential by creating public charter community schools that provide a 
high-quality schooi of choice with curriculum programs and instructional methods that 
are specifically designed for the student popuiation. 

1998 



Grade Levels Served: 



K-11 



Number of Schools 
Total: 



Urban: 



Suburban: 



Rural: 



CM 

CVl 

CO 


N/A 




N/A 


N/A 




Costs 


Total Operating 
Costs: 


Training: 


Materials: 


Personnel: 


Other: 


Yearl 


N/A 


N/A 


N/A 


N/A 


N/A 


Year 2 


N/A 


N/A 


N/A 


N/A 


N/A 


Year 3 


N/A 


N/A 


N/A 


N/A 


N/A 


Years 4+ 


N/A 


N/A 


N/A 


N/A 


N/A 



1 . Evidence of Positive Effects on Student Achievement: 

a. Overall effects 

b. Evidence of positive effects for diverse student populations 

c. Evidence of positive effects in subject areas 

2. Evidence of Positive Effects on Additional Outcomes 

3. Evidence of Positive Effects on Parent, Family, and Community Involvement 

4. Evidence of Link Between Research and the Model’s Design 

5. Evidence of Services and Support to Schools to Enable Successful Implementation: 

a. Evidence of readiness for successful implementation 

b. Evidence of professional development/technical assistance for successful implementation 



0 

® 

® 

(nrI 



) = Very Strong ^ = Moderately Strong 3 = Moderate = Limited 0 = Zero 0 = Negative (0 = No Rating 



This description is based on publicly available information, including the model’s Web site, regarding the model and its 
costs. The Comprehensive School Reform Quality Center attempted to obtain specific information, but this was not always 
possible. Areas in which exact information was not provided are marked by “N/A.” 



'white Hat Management offers many different educational services; however, this report focuses solely on HOPE Academies, because they are 
the only K-12 public education component of White Hat that focuses on a general student population. 

^White Hat currently operates 13 schools as HOPE Academies. 



WHITE HAT MANAGEMENT (HOPE ACADEMIES) 



M odel Description 

All of the information on White Hat Management 
and the HOPE Academies model was collected using 
(a) White Hat’s Weh site and (h) responses from con- 
versations with individuals representing two charter 
authorizers who oversee the charter authorization 
process for Hope Academies. Although contact was 
initiated with a representative of White Hat, the 
Comprehensive School Reform Quality (CSRQ) 

Center was unahle to conduct a conversation with the 
model provider. For more specific information on the 
HOPE Academies model, sites should directly contact 
White Hat Management, the model’s developer. 

According to the provider’s Weh site, David L. Brennan, 
founder and chairman of White Hat, began his work 
in education reform hy creating employee learning 
centers at his manufacturing company in the 1980s. 
When he saw the positive effect of education for his 
own employees, he expanded his focus to include K-12 
students and founded White Hat as an education service 
provider (ESP) in 1998. White Hat offers many differ- 
ent education services to varying student populations, 
including home-schooled students, high school 
dropouts, and distance-learning students. For the 
purpose of this report, the CSRQ Center focuses on 
HOPE Academies, White Hat’s charter school model 
that serves students in grades K-11. White Hat pro- 
vides comprehensive management and education 
services to all HOPE Academies. 

According to the CSRQ Center’s standards, the follow- 
ing were identified as core components of White 
Hat’s HOPE Academies: organization and governance; 
professional development; curriculum; instruction; 
inclusion; technology; student assessment; data-hased 
decision making; and parent, family, and community 
involvement. Core components are considered essential 
to successful implementation. 



Mission/Focus 

According to the model’s Weh site, the vision statement 
of HOPE Academies is that there is hope for every 
child. HOPE Academies claim to help students reach 
their academic potential hy creating public charter 
schools that provide high-quality curriculum programs 
and instructional methods that are specifically designed 
for the student population. The ESP states that it helps 
parents and children overcome obstacles that may 
impede students’ academic progress. 

To accomplish this vision, the provider emphasizes 
seven guiding principles for working with public 
charter schools. These seven principles are known as 
the White Hat Way: 

■ Tailoring instruction to meet the needs of each 
student 

■ Focusing on the fundamentals of reading, writing, 
and math 

■ Integrating technology to supplement instruction 
in the classroom 

■ Employing passionate teachers 

■ Using data-driven decisionmaking 

■ Involving parents actively in their child’s educational 
progress 

■ Designing each school around the community it 
serves 

Goals/Rationale 

According to the provider’s Web site, HOPE Academies 
establishes six goals for each student in its schools: 

1. Each student will be motivated, taught, guided, and 
supported as he/she acquires the desire for and 
accepts the challenge of a lifetime of learning. 
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2. Each student will enjoy success as the natural out- 
come of his/her exploratory behavior and as a 
result, will become an active learner who partici- 
pates in his/her own educational process. 

3. Each student will receive instruction in a manner 
that results in the highest level of learning for 
that child. 

4. Each student will learn ways to influence his/her 
school environment in a positive manner. 

5. Each student will grow in positive self-confidence, 
have the ability to solve problems and make deci- 
sions, and display responsible behavior. 

6. The parents and guardians of students will be 
actively involved in and supportive of their child’s 
educational progress. 



C osts 

The CSRQ Center did not conduct a conversation 
with the ESP, nor was it able to find publicly available 
information regarding the model’s costs. Eor specific 
information on the costs of training, materials, and 
personnel, sites should directly contact White Hat. 



E vidence of Positive Effects on Student 
Achievement 

Evidence of Positive Overall Effects 
Rating: 0 

The CSRQ Center reviewed one quantitative study 
of White Hat on student achievement but this study 
did not have sufficient rigor to meet the CSRQ 
Center’s standards. Therefore, the overall rating of the 
evidence of positive effects of this model on student 
achievement is zero. (Appendix G reports on the 



study that was reviewed but did not meet CSRQ 
Center standards.) 

Evidence of Positive Effects for Diverse Student 
Populations 

Rating: ® 

Because no studies of White Hat met the CSRQ Center’s 
standards for review, the impact of this model on stu- 
dent achievement for diverse populations is unknown. 
Therefore, the rating in this subcategory is no rating. 

Evidence of Positive Effects in Subject Areas 
Rating: ® 

Because no studies of White Hat met the CSRQ Center’s 
standards for review, the rating in this subcategory is 
no rating. 

E vidence of Positive Effects on 
Additional Outcomes 

Rating: 

Because no studies of White Hat met the CSRQ Center’s 
standards for review, it was not able to evaluate the 
effects of White Hat on additional outcomes. 
Therefore, the rating in this category is no rating. 



E vidence of Positive Effects on Parent, 
Eamily, and Community Involvement 

Rating: 

No studies that met the CSRQ Center’s standards 
examined the effects of White Hat on parent, family, 
and community involvement. Therefore, the rating in 
this category is no rating. 
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E vidence of Link Between Research and 
the Model’s Design 

Rating: @ 

The CSRQ Center did not conduct a conversation with 
the ESP, nor was it able to find publicly available infor- 
mation to allow it to rate this dimension. Consequently, 
the model received no rating in this category. 



E vidence of Services and Support to 
Schools to Enable Successful Implementation 

Evidence of Readiness for Successful Implementation 
Rating: ® 

The CSRQ Center did not conduct a conversation with 
the ESP, nor was it able to find publicly available infor- 
mation to allow it to rate this dimension. Consequently, 
the model received no rating in this subcategory. 

Evidence of Professional Development/Technical 
Assistance for Successful Implementation 

Rating: ® 

The CSRQ Center did not conduct a conversation with 
the ESP, nor was it able to find publicly available infor- 
mation to allow it to rate this dimension. Consequently, 
the model received no rating in this subcategory. 



C entral Components 



Administrative Services 

White Hat states on its Web site that it provides com- 
prehensive administrative services to its school. The 
provider offers information technology consulting and 
support to determine how schools can best use modern 



technology inside and outside the classroom. The 
provider also offers marketing and communications 
services, such as producing annual school reports for 
each HQPE Academy that details student performance, 
attendance rates, and student and parent satisfaction. 
The provider offers many other general operations 
management services, including reporting student 
data to state education authorities, providing school 
safety resources, negotiating contracts for different 
school products and services, and grant writing support 
that helps to identify and obtain federal and state 
grants and special federal program funding. 

White Hat also provides comprehensive human 
resource services, such as recruiting new employees, 
creating organization charts and job descriptions and 
staffing requirements to hire new employees, developing 
a school’s policies and procedures, and offering health 
care and retirement benefits for employees. In addition. 
White Hat provides complete financial management 
services, such as accounting, preparation of financial 
statements, creation and monitoring of school budgets, 
internal audits, and assistance for external audits. As a 
subsidiary of White Hat, White Hat Realty provides 
facilities management by obtaining, designing, con- 
structing, and maintaining appropriate school sites. 
Lastly, White Hat provides assistance to those interested 
in creating new charter schools. 

Organization and Governance 

Each HQPE Academy, as a public charter school, has a 
board of directors and a charter authorizer that oversees 
its progress. HQPE Academy High School Campus 
extends the mission and goals of the HQPE Academies 
to serve students in grades 9-11. White Hat, according 
to its Web site, plans to extend the high school program 
to 12th grade in 2006. 

The ESP, according to its Web site, emphasizes a safe, 
clean, and positive school environment. To this end, 
teachers, staff, and students wear uniforms. Students 
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are expected to respond “yes madam” or “yes sir” to 
build a culture of respect. 

Curriculum and Instruction 

HOPE Academies focus on reading and math as the 
foundation for academic success. The provider indi- 
cates that the curriculum is designed to help students 
master reading, writing, and math in the primary 
grades. Other subjects, such as science, social studies, 
and art, are added to the fundamental focus on reading 
and math in middle and high school grades. 

The model’s Web site states that HOPE Academies use 
research-based practices that are aligned with state 
standards and supported by experts in the curriculum 
development field. In kindergarten through third grade, 
HOPE Academies teach reading using the Direct 
Instruction (DI) method.^ DI methods provide a sys- 
tematic approach that allows each student to master 
one component or skill at his/her individual pace 
before moving on to the next component or skill. 

In addition, HOPE Academies use the Calvert Reading 
Curriculum, a spiraling curriculum that focuses on 
skill development at all grade levels. Eor students in 
grades K-2, this curriculum emphasizes phonics 
development, decoding, and comprehension using 
phonics story books and anthologies. Students in 
grades 3-5 are exposed to classic childrens literature 
to develop their reading skills in interpretation, appre- 
ciation, analysis, and critical thinking. In middle school, 
teachers use poetry and the classics to teach reading 
and develop critical thinking skills. 

Along with the curricular standards set by the provider. 
White Hat also states that individualized instruction is 
one of the provider’s guiding principles. The model 
provider indicates that HOPE Academies develop a 



specific educational plan for each student based on 
his/her individual needs. 

According to the provider’s Web site, special needs 
students are serviced through a local service delivery 
plan and in accordance with state regulations. A special 
education resource teacher or consultant is assigned to 
each HOPE Academy to help ensure that the local 
service delivery plan is carried out. In addition. White 
Hat indicates that assistance is provided to students 
with limited English proficiency through a combina- 
tion of resources that include the Ohio Department 
of Education’s English as a Second Language Bilingual 
and Multicultural Center, the Special Education 
Regional Resource Center and, when possible, a 
teacher of English as a second language. 

Scheduling and Grouping 

White Hat does not have specific scheduling or group- 
ing requirements for its schools. Typically, a HOPE 
Academy’s school year is aligned with the district’s 
schedule, starting in late August and ending in early 
June. The hours of operation for HOPE Academies 
vary by school location. 

Technology 

One of White Hat’s guiding principles is using the latest 
technology to supplement instruction. According to 
the provider, every teacher’s desk has a computer with 
Internet access. Each HOPE Academy also has 
approximately six computers in all classrooms available 
for student use and a larger computer lab with approxi- 
mately 30 computers. To help students use technology 
more effectively, every HOPE Academy has a fuU-time 
technology teacher or technology consultant. Specifically 
at the high school level, HOPE Academies aim to pro- 
vide one laptop computer to every student. 



^The CSRQ Center reviewed DI when used as a full immersion comprehensive school reform model in an earlier report: CSRQ Center Report on Elementary 
School Comprehensive School Reform Models. The report can be accessed at http://www.csrq.org. 
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In addition, students can opt to apply for the HOPE 
Extended Learning Program. This Weh-hased program 
allows students to receive assignments, submit home- 
work, and access tutors especially when the student is 
away from a structured classroom environment (e.g. 
summer break, holidays, or illnesses). 

According to the ESP, traditional teaching is supple- 
mented with curriculum software that integrates 
coursework and curriculum objectives. Eor specific 
information on costs and use of software, sites should 
directly contact White Hat. 

One charter authorizer commented in a conversation 
with the CSRQ Center that one of White Hat’s strongest 
components is its effective use of technology in the 
classroom. 

Monitoring Student Progress and Performance 

White Hat indicates that one of its guiding principles 
is measuring and analyzing all educational outcomes 
to best serve its students. HOPE Academies use several 
testing instruments to gauge student progress. Because 
HOPE Academies are currently located only in Ohio, 
students take the Ohio Proficiency tests as required by 
state law. HOPE Academies also use the Iowa Cognitive 
Ability Tests as a diagnostic assessment tool, with the 
goal to individualize instruction based on students’ 
academic progress. 

According to the provider. White Hat surveys students 
and parents twice per year. Results of these surveys, 
attendance rates of students and teachers, and the 
results from the Ohio state assessment are published 
for each school in an Annual Progress Report. 

One charter authorizer noted that White Hat uses the 
Authorizer Oversight Information System, a data 
management system developed by Central Michigan 
University, to track the academic performance of each 
of its schools. The system uses both raw scores and 



value-added outcome measures to demonstrate the 
progress of students in HOPE Academies. 

Family and Community Involvement 

White Hat indicates that one of its guiding principles 
is to involve parents actively in their child’s educational 
progress. To meet this goal, the provider encourages 
all schools to solicit parental input for student progress. 
In addition, all HOPE Academies have family advocates 
or trained counselors onsite that, according to the 
provider, form relationships with students, family, 
staff, and the community. 

One charter authorizer of HOPE Academies told the 
CSRQ Center that one of White Hat’s strengths is 
providing wrap-around services, such as information 
sessions for parents on community services available 
to them, that could further strengthen the family’s 
living situation. 

Professional Development and Technical Assistance 

According to the provider. White Hat provides 3 weeks 
of in-service training to all new teachers in HOPE 
Academies. White Hat also offers ongoing training 
and learning opportunities for all staff with a focus on 
professional development for specific curricula and 
instructional methods, such as DI and Calvert Reading. 
One charter authorizer, in a conversation with the 
CSRQ Center, noted that coaches visit different class- 
rooms on a weekly basis to provide feedback and 
proper modeling of DI. 

Implementation Expectations/Benchmarks 

The CSRQ Center did not conduct a conversation 
with the ESP, nor was it able to find publicly available 
information regarding the model’s benchmarks. 
However, conversations with charter authorizers indi- 
cated that White Hat provides each HOPE Academy 
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with a set of benchmarks. For more specific information 
on timelines and benchmarks for implementation, 
sites should directly contact the ESP. 

Special Considerations 

HOPE Academies distributes a Home Resource Guide to 
every student. This printed guide provides resources to 
support classroom instruction and to enable students 
to complete self-directed home study and research 
projects. Suggested resources include Web sites, games, 
activities, television programs, grade-appropriate 
books, and tips for parents. The resource guide is also 
available through the ESP’s Web site. 

As stated previously, the CSRQ Center did not conduct 
a conversation with model provider. All information 
provided is based on the model’s Web site and other 
publicly available materials. In addition, conversations 
with two charter authorizers provided feedback on the 
nature of White Hat’s services at individual school 
sites. Por more specific information on White Hat and 
HOPE Academies, schools should directly contact the 
model provider. 
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Appendix A: Edison Schoois 



The following is a description of studies that did not 
meet the Comprehensive School Reform Quality 
(CSRQ) Center’s standards. 

Nineteen studies of Edison Schools did not meet the 
CSRQ Center’s standards. Qf those studies, 8 were 
eligible for full review because they used quasi- 
experimental or longitudinal research designs. Their 
findings, however, were considered to be inconclusive 
for the following reasons: Two studies did not control 
for preexisting differences between the treatment and 
comparison groups, and six studies did not provide 
sufficient evidence that the model was implemented 
as intended. 

The remaining 1 1 studies were ineligible for full review 
for the following reasons: Three studies did not con- 
tain baseline measures on which to establish equiva- 
lence between the groups; one study was descriptive 
and did not include quantitative data; two studies did 
not have control groups to compare Edison students; 
three studies were not eligible for further review 
because a more recent version of the study was avail- 
able; finally, the reported results in two studies com- 
bined models from more than one education service 
provider, making it impossible to attribute any of the 
results solely to Edison. 



ot Eligible for Initial Review 

Cookson, R W., Embree, K., & Eahey, S. (2000). The 
Edison Partnership schools: An assessment of 
academic climate and classroom culture. New 
York: Teachers College, Columbia University. 

Hoxby, C. M. (2003). School choice and school 

competition: Evidence from the United States. 
Swedish Economic Policy Review, 10, 9-65. 




Not Relevant for Initial Review: Study was not quantitative 
or on a comprehensive schooi reform model or on 
elementary school students. 

Not Eligible for Full Review: Study’s research design 
was not sufficiently rigorous or did not include student 
achievement outcomes. 

Did Not Meet Standards (Inconclusive): Study had critical 
threats to causal validity. 

Met Standards (Suggestive): Study had no critical threats 
to validity but used a less rigorous (e.g., longitudinal, 
cohort) research design. 

Met Standards (Conclusive): Study had no critical threats 
to validity and used a rigorous (e.g., experimental, quasi- 
experimental) research design. 



Miller, L. A. (2000). The impact of the Edison design 
on teachers and their perceptions of its impact on 
improved student achievement over time. 
Dissertation Abstracts International, 6i (07-A), 
2542. 

Rhim, L. M. (2002). School privatization by way of a 
comprehensive management contract: A single case 
study of the extent to which privatization theory 
transfers to practice in a public charter school. 
Paper presented at the Annual Meeting of the 
American Educational Research Association, 
New Qrleans, LA. 

US. General Accounting Qffice. (2002). Public schools: 
Insufficient research to determine effectiveness of 
selected private education companies (GAQ-03-11). 
Washington, DG: Author. 
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ot Eligible for Full Review 

American Federation of Teachers. (2000). Trends in 
student achievement for Edison Schools, Inc.: The 
emerging track record. Washington, DC: Author. 

Dryden, M. (2002). Achievement performance report: 
Dallas-Edison Partnership schools 2001 -2002. 
Dallas, TX: Division of Evaluation and 
Accountability, Dallas Independent School 
District. 

Edison Project/Wichita Public Schools. (1997). End-of- 
year report on Dodge Edison Elementary School. 
New York: Edison Schools. 

Hoxby, C. M., & Rockoff, J. E. (2004). The impact of 
charter schools on student achievement: A study 
of students who attend schools chartered by the 
Chicago Charter School Eoundation. Cambridge, 
MA: Harvard University. 

Minneapolis Public Schools. (2001). Edison High: School 
information report 2000-2001. Minneapolis, MN: 
Author. 

Minneapolis Public Schools. (2001). Edison/PPL 

Elementary: School information report 2000-2001. 
Minneapolis, MN: Author. 

Miron, G. (2004). Evaluation of the Delaware Charter 
school reform, year 1 report. Kalamazoo, ME 
The Evaluation Center, Western Michigan 
University. Retrieved November 14, 2005, 
from http://www.doe.kl2.de.us/files/pdf/ 
dedoe_charterschreform2004.pdf 

Nelson, E. H., & Van Meter, N. (2003). Update on 
student achievement for Edison Schools, Inc. 
Washington, DC: American Eederation of 
Teachers. 




Shay, S. A. (2000). A longitudinal study of achievement 
outcomes in a privatized public school: A growth 
curve analysis. Dissertation Abstracts International, 
6i(05-A), 1746. 

Useem, E., & Balfanz, R. (2002, Winter). Comprehensive 
district reform: Philadelphia's grand experiment. 
Benchmarks, 4(1). 

US. General Accounting Office. (2003). Public schools: 
Comparison of achievement results for students 
attending privately managed and traditional 
schools in six cities (GAO-04-62). Washington, 
DG: Author. 




id Not Meet Standards (Inconclusive) 



Gill, B. P, & Edison Schools. (2005). Seventh annual 
report on school performance 2003-2004. 
Arlington, VA: RAND Gorporation. 

Edison Schools. (2000). Third annual report on school 
performance. New York: Author. 



Miron, G., Nelson, G., & Risley, J. (2002). Strengthening 
Pennsylvania's charter school reform: Pindings 
from the statewide evaluation and discussion of 
relevant policy issues. Kalamazoo, ME The 
Evaluation Genter, Western Michigan University. 
Retrieved November 22, 2005, from 
http://www.wmich.edu/evalctr/charter/pa_5year/ 



Mislevy, R. J. (1996). Reading achievement test-score 
analysis: 1995/96 King-Edison vs. control schools, 
grades K-2, Mount Clemens Community Schools. 
Retrieved January 24, 2006, from 
http://web.archive.Org/web/19990209120006/ 
www.aft.org/research/edisonproject/sfa/w96/ 
mislevy.htm 



RAND Gorporation. (2001). Edison Schools 2000-2001 
annual report on school performance. Arlington, 
VA: Author. 
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RAND Corporation. (2002). Edison Schools 2001-2002 
annual report on school performance. Arlington, 
VA: Author. 

RAND Corporation. (2003). Edison Schools 2002-2003 
annual report on school performance. Arlington, 
VA: Author. 

Rhim, L. M. (2005). Restructuring schools in Chester 
Upland, Pennsylvania: An analysis of state 
restructuring efforts. Denver, CO: Education 
Commission of the States. 



M et Standards (Suggestive) 

American Federation of Teachers. (1998). Student 
achievement in Edison Schools: Mixed results in 
an ongoing enterprise. Washington, DC: Author. 

Miron, G., & Applegate, B. (2000). An evaluation of 
student achievement in Edison schools opened in 
1995 and 1996. Kalamazoo, MI: The Evaluation 
Center, Western Michigan University. 



M et Standards (Conclusive) 

Dryden, M. (2004). The performance of Edison Schools 
Inc. in the Dallas schools. Paper presented at the 
Annual Meeting of the American Educational 
Research Association, San Diego, CA. 

GUI, B. R, Hamilton, L. S., Lockwood, J. R., Marsh, J. 
A., Zimmer, R. W, Hill, D., et al. (2005). 
Inspiration, perspiration, and time: Operations 
and achievement in Edison Schools. Arlington, 
VA: RAND Corporation. 

Miron, G. (2006). Evaluation of the Delaware Charter 
school reform, year 2 report. Kalamazoo, MI: 

The Evaluation Genter, Western Michigan 
University. Retrieved February 21, 2006, 



from http://www.doe.kl2.de.us/files/pdf/ 
dedoe_charterschreform2006.pdf 

Mislevy, R. J. (1996). Reading achievement test-score 
analysis: 1995/96 Washington- Edison School, 
grades K-2 Sherman Independent School District. 
Retrieved January 24, 2006, from 
http://weh.archive.Org/weh/19990209120006/ 
www.aft.org/research/edisonproject/sfa/w96/ 
mislevy.htm 

Mislevy, R. J. (1997). Reading achievement test-score 
analysis: 1996/97 Dodge-Edison vs. control 
schools, grades 1 -3 Wichita Unified School 
District #259. Retrieved January 24, 2006, from 
http://weh.archive.Org/weh/19990209120006/ 
www.aft.org/research/edisonproject/sfa/w96/ 
mislevy.htm 

Mislevy, R. J. (1997). Reading achievement test-score 
analysis, 1 996/97, grades K-2, Roosevelt-Edison 
School, Colorado Springs, CO. Retrieved January 
24, 2006, from http://weh.archive.org/weh/ 
19990209120006/www.aft.org/research/ 
edisonproject/sfa/w96/mislevy.htm 

Shay, S. A., & Gomez, J. J. (2002). Privatization in 

education: A growth curve analysis of achievement. 
Paper presented at the Annual Meeting of the 
American Educational Research Association, 
New Orleans, LA. 
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Appendix B: Imagine Schoois 



No studies of Imagine Schools were available for 
review by the Comprehensive School Reform Quality 
(CSRQ) Center. 



Not Relevant for Initial Review: Study was not quantitative 
or on a comprehensive schooi reform model or on 
elementary school students. 

Not Eligible for Full Review: Study’s research design 
was not sufficiently rigorous or did not include student 
achievement outcomes. 

Did Not Meet Standards (Inconclusive): Study had critical 
threats to causal validity. 

Met Standards (Suggestive): Study had no critical threats 
to validity but used a less rigorous (e.g., longitudinal, 
cohort) research design. 

Met Standards (Conclusive): Study had no critical threats 
to validity and used a rigorous (e.g., experimental, quasi- 
experimental) research design. 
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Appendix C: The Leona Group, L.L.C 



The following is a description of studies that did not 
meet the Comprehensive School Reform Quality 
(CSRQ) Center’s standards. 

The two studies of The Leona Group, L.L.C. that were 
reviewed did not meet the CSRQ Center’s standards. 
Qne study was eligible for full review but was consid- 
ered to be inconclusive. This study followed a Leona 
school over time, but the study did not meet the stan- 
dards for establishing an adequate baseline measure. 

The second study was not eligible for full review 
because it did not use a rigorous research design. This 
study provided no comparison group or baseline year; 
it looked only at the treatment group’s posttest scores. 




ot Eligible for Full Review 



The Leona Group, L.L.G. (2004). 2003-2004 educational 
report. East Lansing, MI: Author. 



Not Relevant for Initial Review: Study was not quantitative 
or on a comprehensive schooi reform model or on 
elementary school students. 

Not Eligible for Full Review: Study’s research design 
was not sufficiently rigorous or did not include student 
achievement outcomes. 

Did Not Meet Standards (Inconclusive): Study had critical 
threats to causal validity. 

Met Standards (Suggestive): Study had no critical threats 
to validity but used a less rigorous (e.g., longitudinal, 
cohort) research design. 

Met Standards (Conclusive): Study had no critical threats 
to validity and used a rigorous (e.g., experimental, quasi- 
experimental) research design. 




id Not Meet Standards (Inconclusive) 



Horn, J., & Miron, G. (2000). An evaluation of the 
Michigan Charter School initiative: Performance, 
accountability, and impact. Kalamazoo, MI: The 
Evaluation Genter, Western Michigan University. 



NOT ELIGIBLE FOR FULL REVIEW 
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Appendix D: Mosaica Education 



The following is a description of studies that did not 
meet the Comprehensive School Reform Quality 
(CSRQ) Center’s standards. 

Three studies of Mosaica Education were reviewed but 
did not meet CSRQ Center standards.^ Qf those, one 
study was eligible for full review but did not provide 
sufficient evidence that the model was implemented 
as intended. Therefore, the study was considered to be 
inconclusive. 

The remaining two studies were ineligible for full 
review. Qne of these studies intended to test the 
impact of Mosaica, but the research design was not 
eligible for full review because it did not include a 
control group that could be compared with the treat- 
ment group. The second study was not an evaluation 
of the impact of Mosaica. Instead, the study was 
more descriptive in nature than evaluative. 



ot Relevant for Initial Review 

Cross, R. W., Rebarber, T, & Wilson, S. E (2002). 

Student gains in a privately managed network of 
charter schools using Direct Instruction. Journal 
of Direct Instruction, 2(1): 3-21. 




Not Relevant for Initial Review: Study was not quantitative 
or on a comprehensive schooi reform model or on 
elementary school students. 

Not Eligible for Full Review: Study’s research design 
was not sufficiently rigorous or did not include student 
achievement outcomes. 

Did Not Meet Standards (Inconclusive): Study had critical 
threats to causal validity. 

Met Standards (Suggestive): Study had no critical threats 
to validity but used a less rigorous (e.g., longitudinal, 
cohort) research design. 

Met Standards (Conclusive): Study had no critical threats 
to validity and used a rigorous (e.g., experimental, quasi- 
experimental) research design. 



http://www.doe.kI2.de.us/files/pdf/ 

dedoe_charterschreform2006.pdf 

Nelson, E. H., & Van Meter, N. (2003). Student achieve- 
ment in schools managed by Mosaica. Washington, 
DC: American Eederation of Teachers. 




id Not Meet Standards (Inconclusive) 



ot Eligible for Full Review 

Miron, G. (2006). Evaluation of the Delaware Charter 
School reform, year 2 report. Kalamazoo, MI: 
The Evaluation Center, Western Michigan 
University. Retrieved November 9, 2005, from 




Miron, G., Nelson, G., & Risley, J. (2002). Strengthening 
Pennsylvania's charter school reform: Findings 
from the statewide evaluation and discussion of 
relevant policy issues. Kalamazoo, MI: The 
Evaluation Genter, Western Michigan University. 
Retrieved November 9, 2005, from 
http://www.wmich.edu/evalctr/charter/pa_5year/ 



'An additional study was not available in full copy for a review by the CSRQ Center and therefore, is not included in this report: Cash, R. W. (2002). Mosaica 
Education annual report: Testing results 1 998-2002. San Francisco: WestEd. 
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M et Standards (Conclusive) 

Miron, G. (2004). Evaluation of the Delaware Charter 
School reform, year 1 report. Kalamazoo, MI: 
The Evaluation Center, Western Michigan 
University. Retrieved November 9, 2005, from 
http://www.doe.kl2.de.us/files/pdf/ 
dedoe_charterschreform2004.pdf 



MET STANDARDS (CONCLUSIVE) 
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Appendix E: National Heritage Academies 



The following is a description of studies that did not 
meet the Comprehensive School Reform Quality 
(CSRQ) Center’s standards. 

Three studies of National Heritage Academies (NHA) 
that were reviewed did not meet the CSRQ Center’s 
standards. Qf those, one was eligible for full review but 
was considered to be inconclusive. This study followed 
NHA schools over time, but it did not establish an 
adequate baseline measure. 

The two remaining studies were not eligible for full 
review because they did not use rigorous research 
designs. Qne study examined pretest-posttest changes 
without using a comparison group. The other study 
provided no comparison group or baseline year and 
looked only at the treatment group’s posttest scores. 



ot Eligible for Full Review 

Hess, R M., & Leal, D. L. (2003). An evaluation of stu- 
dent achievement in National Heritage Academy 
charter schools, 2000-2003. Washington, DC: 
American Enterprise Institute. 




Not Relevant for Initial Review: Study was not quantitative 
or on a comprehensive schooi reform model or on 
elementary school students. 

Not Eligible for Full Review: Study’s research design 
was not sufficiently rigorous or did not include student 
achievement outcomes. 

Did Not Meet Standards (Inconclusive): Study had critical 
threats to causal validity. 

Met Standards (Suggestive): Study had no critical threats 
to validity but used a less rigorous (e.g., longitudinal, 
cohort) research design. 

Met Standards (Conclusive): Study had no critical threats 
to validity and used a rigorous (e.g., experimental, quasi- 
experimental) research design. 



Wolfram, G. L. (2002). Making the (better) grade: A 
detailed statistical analysis of the effect of National 
Heritage Academies on student MEAP scores. 
Hillsdale, MI: Hillsdale Policy Group, Ltd. 




id Not Meet Standards (Inconclusive) 



Horn, J., & Miron, G. (2000). An evaluation of the 
Michigan Charter School initiative: Performance, 
accountability, and impact. Kalamazoo MI: The 
Evaluation Genter, Western Michigan University. 



NOT ELIGIBLE FOR FULL REVIEW 
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Appendix F: SABIS Educational Systems, Inc. 



No studies of SABIS were available for review by the 
Comprehensive School Reform Quality (CSRQ) Center. 



Not Relevant for Initial Review: Study was not quantitative 
or on a comprehensive schooi reform model or on 
elementary school students. 

Not Eligible for Full Review: Study’s research design 
was not sufficiently rigorous or did not include student 
achievement outcomes. 

Did Not Meet Standards (Inconclusive): Study had critical 
threats to causal validity. 

Met Standards (Suggestive): Study had no critical threats 
to validity but used a less rigorous (e.g., longitudinal, 
cohort) research design. 

Met Standards (Conclusive): Study had no critical threats 
to validity and used a rigorous (e.g., experimental, quasi- 
experimental) research design. 
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Appendix G: White Hat Management 



The one study of White Hat Management that was 
reviewed was not eligible for full review because it did 
not meet the Comprehensive School Reform Quality 
(CSRQ) Center’s standards. The study did not use 
rigorous research designs, using a matched comparison 
of treatment and control schools with no pretest to 
establish equivalence. 



ot Eligible for Full Review 

Ohio State Legislative Office of Education Oversight. 
(2002). Community schools in Ohio: Preliminary 
report on proficiency test results, attendance, and 
satisfaction. Columbus, OH: Author. 




Not Relevant for Initial Review: Study was not quantitative 
or on a comprehensive schooi reform model or on 
elementary school students. 

Not Eligible for Full Review: Study’s research design 
was not sufficiently rigorous or did not include student 
achievement outcomes. 

Did Not Meet Standards (Inconclusive): Study had critical 
threats to causal validity. 

Met Standards (Suggestive): Study had no critical threats 
to validity but used a less rigorous (e.g., longitudinal, 
cohort) research design. 

Met Standards (Conclusive): Study had no critical threats 
to validity and used a rigorous (e.g., experimental, quasi- 
experimental) research design. 



NOT ELIGIBLE FOR FULL REVIEW 
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Appendix H: Letters From Model Providers 



On March 1, 2006, the Comprehensive School Reform Quality (CSRQ) Center provided all model developers 
with background information on this report and with an opportunity to comment on the accuracy of the CSRQ 
Center’s review of the education service provider (ESP). In most instances, this contact was a followup to ongoing 
communication with the ESPs throughout the development of this report. 

The CSRQ Center invited ESPs to share questions and concerns about the reviews and provide documentation for 
any information they needed to be corrected. Several ESPs engaged in telephone and e-mail communication with the 
CSRQ Center to provide valuable insight and information on improving the report. The CSRQ Center considered 
all concerns and suggested edits for the final narrative. 

The CSRQ Center also encouraged providers to submit a two-page letter about the review of their models that 
could be published along with the report. The letters received from the ESPs give consumers additional informa- 
tion that they can consider in making decisions about adopting a model. The following ESPs submitted letters 
of comment: 

■ The Leona Group, L.L.C. 



All letters have been reproduced as submitted to the CSRQ Center. 
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APPENDIX H: LETTERS FROM MODEL PROVIDERS 



L etter From The Leona Group, L.L.C. 

(Reproduced As Submitted) 



The Leona Group, L.L.C, 

A new kind of public school 

Arizona ■ Indiana * Michigan * Ohio 



Response to CSRQ Center: 



The most important factor to note about The Leona Group is that it does not prescribe a rigid curriculum model 
or design for the schools it manages. Each school staff uses best practices and methodologies, following state 
standards and benchmarks, which best fit its students' needs. Based on our experience and philosophy of 
education, we disagree that one model/size fits all. 

The Leona Group is unlike the majority of educational service providers managing charier schools in that it 
does not mandate a particular curriculum model. This is a key reason why community boards choo.se Leona to 
manage their schools. Leona does not subscribe to a cookie-cutter teaching model because its flexibility in 
curriculum design allows each school staff to quickly and innovatively make decisions for students, many of 
whom are at least two grade levels behind their peers when they enter a Leona school. Our .schools offer a rich 
variety of proven educational models. 

Because of a highly focused Quality Schools Initiative begun three years ago. the majority of .schools managed 
by The Leona Group met Adequate Yearly Progress academic performance measures in 2005. This is due to a 
strong emphasis on professional development for staff and the addition of a curriculum coach to the school 
leadership team. This person assists the school leader and teachers in ensuring that school's curriculum, pacing 
guides, lesson plans and assessments are effectively in place and aligned with state standards. All teachers 
receive an annual formal performance review along with several informal evaluations throughout the school 
year. This Leona initiative has been praised by outside consultants and state education department officials as a 
“model” for other public schools. Students are making academic progress because of this strategic initiative. 
Their performance data are readily available from annual education reports that are at the schools and posted to 
the Internet. 

In addition to high academic achievement and focused professional training. Leona schools have strong parental 
support; many schools have waiting lists. This is the type of model that Leona provides to its sehools. Because 
there is no specific or singular educational model that is mandated to Leona-managed .schools, it is not correct 
and is in fact misleading to rate or review The Leona Group against a methodology regarding a singular 
company wide curriculum model. Instead of providing specific ratings for our organization, it is accurate to list 
The Leona Group as Does Not Fit CSRQ Center Methodology. 



www.leonagroup. com 

1313 E. Osborn Rd Suite 100 Phoenix, Arizona 85014 T 602.953.2933 F 602 953 0831 
4660 S. Hagadorn Rd Suite 500 East Unsing, Michigan 48823 T 517.333.9030 F 517.333 4559 
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Appendix I: Study Findings Summary Tabies 



Table 1-1 . Quantitative Study Findings Used to Rate Evidence of 
Overall Positive Effects on Student Achievement 



Education Service Provider 


Initially 

Relevant 


Number of Studies 
Eligible for Meeting 

Full Review Standards Conclusive 


Suggestive 


Number of 
Findings 


Percentage 
of Positive 
Findings 


Edison Schools 


28 


17 


9 


7 


2 


19 


40 


Imagine Schools 


0 


0 


0 


0 


0 


N/A 


N/A 


The Leona Group, L.L.C. 


2 


1 


0 


0 


0 


N/A 


N/A 


Mosaica Schools 


3 


1 


0 


0 


0 


N/A 


N/A 


National Heritage Academies 


3 


1 


0 


0 


0 


N/A 


N/A 


SABIS Educational 
Systems, Inc. 


0 


0 


0 


0 


0 


N/A 


N/A 


White Hat Management 
(HQPE Academies) 


1 


0 


0 


0 


0 


N/A 


N/A 


TOTAL 


37 


20 


9 


7 


2 


19 





Key: 

Initially Relevant: Of the nearly 940 studies screened, the number of studies per model found to be relevant to this review. 

Eligible for Full Review: The number of studies per model that used research designs that were sufficiently rigorous and included student 
achievement outcomes. 

Meeting Standards: The number of studies per model considered to be suggestive or conclusive according to causal validity rubrics of 
the CSRQ Center’s Quality Review Tool. 

Conclusive: The number of studies per model that used a rigorous research design (e.g., experimental, quasi-experimental) with no critical 
threats to validity. 

Suggestive: The number of studies per model that used a less rigorous research design (e.g., longitudinal) with no critical threats to validity. 
Number of Findings: The total number of individually measured outcomes found in the studies that met the CSRQ Center’s standards. 
The “N/A' indicates models in which zero studies met the CSRQ Center’s standards. 

Percentage of Positive Findings: The percentage of total findings in the studies that met the CSRQ Center’s standards that were statisti- 
cally significant and indicated that a model had a positive impact. The “N/A’ indicates models in which zero studies met the CSRQ Center’s 
standards. 
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APPENDIX I: STUDY FINDINGS SUMMARY TABLES 



Table 1-2. Summary of Basic Information by Model 



Education 

Service 

Provider 


Grade 

Levels 

Served 


Number of 
Schools 


Year 

Introduced 
in Schools 


Costs 
(Year 1) 


Evidence of 
Positive 
Overall 
Effects 


Evidence of 
Positive Effects 
for Diverse 
Student 
Populations 


Evidence of 
Positive Effects 
In Subject Areas 


Evidence of 
Positive 
Effects on 
Additional 
Outcomes 


Evidence of 

Positive 

Effects 

on Parent, 

Family, and 

Community 

Involvement 


Evidence of 
Link Between 
Research and 
the Model’s 
Design 


Evidence of 
Readiness for 
Successful 
Implementation 


Evidence of 

Professional 

Development/ 

Technical 

Assistance for 

Successful 

Implementation 


Edison 

Schools 


K-12 


157 


1995 


Varies 


3 


0 ^ 


Reading: (J 
Writing: (0 
Math: 


0 


0 


• 


• 


• 


Imagine 

Schools 


Pre-K-12 


42 


2004 


N/A 


® 


0 


0 


0 


0 


0 


0 


0 


The Leona 
Group, L.L.C. 


Pre-K-12 


48 


1996 


Varies 


0 


0 


0 


0 


0 


3 


3 


3 


Mosaica 

Education 


K-8 


40 


1997 


Varies 


0 


0 


Reading: (0 

Math: (0 


0 


0 


0 


0 


0 


National 

Heritage 

Academies 


K-8 


51 


1995 


Varies 


0 


0 


0 


0 


0 


0 


3 


3 


SABIS 
Educational 
Systems, Inc. 


Pre-K-12 


31^ 


1995 


N/A 


@ 


0 


0 


0 


0 


0 


0 


0 


White Hat 
Management 
(HOPE 
Academies) 


K-11 


32" 


1998 


N/A 


0 


0 


0 


0 


0 


0 


0 


0 



Note. Readers are encouraged to use this table in conjunction with the entire report, which explains in detail how the approaches were reviewed and rated. The report also provides detailed information 
about each model's ratings and offers in-depth descriptions of each model's services. 



'Although the rating in this subcategory is zero, readers should note that most of the studies on Edison that met standards and also demonstrated evidence of positive overall effects on student 
achievement, examined the effects of this model on schools that served primarily low-income and minority populations. 

^SABIS operates 10 schools in the United States. 

''White Hat currently operates 13 schools as HOPE Academies. 
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APPENDIX I: STUDY FINDINGS SUMMARY TABLES 



Table 1-2. Summary of Basic Information by Model (continued) 



Key: 

Grade Levels Served: The grade levels served represents the full range of grades that the model serves. 

Number of Schools: This reflects the number of schools that use the model, as reported by the model provider. This number includes all schools regardless of the length of time implemented 
or the level of implementation. 

Year Introduced in Schools: This date refers to the year in which schools first implemented the model. This is included so that readers can judge whether the ratings are influenced by the 
relative newness of the model. 

Costs (Year 1): The costs are estimates provided by the model provider. The full report provides additional details on costs for each model. 

Evidence of Positive Overall Effects: This rating focuses on a model’s overall effects on student achievement. The rating is a function of the number of studies that were rated to be suggestive 
and conclusive, the percentage of findings in the suggestive and conclusive studies that demonstrated a positive impact, and the average effect size of those findings. The final rating reflects 
the amount of rigorous research and the strength of the effects reported in that research. The full report provides complete information about the methodology used to produce all ratings in 
this report. 

Evidence of Positive Effects for Diverse Student Populations: This rating refers to positive effects for the achievement of students from diverse backgrounds, such as low socioeconomic 
status, minority, special needs, or English language learners. 

Evidence of Positive Effects in Subject Areas: This rating refers to positive effects on achievement in specific subject areas, such as reading, math, writing, science, or social studies. 
Evidence of Positive Effects on Additional Outcomes: This rating refers to positive effects on additional outcomes, such as student discipline, student attendance, school climate, 
retention/promotion rates, or teacher satisfaction. 

Evidence of Positive Effects on Parent, Famiiy, and Community Invoivement: This rating refers to positive effects for improvement in family and community involvement, such as involvement 
in school governance, participation in family nights, or homework support. 

Evidence of Link Between Research and the Model’s Design: This rating refers to evidence that the model developer can provide explicit links between research and the core components 
of the model. Core components are considered essential to successful implementation. 

Evidence of Readiness for Successful Implementation: This rating refers to evidence that the model provider ensures initial commitment from schools, tracks and supports full implementation, 
and helps schools allocate resources for successful implementation. 

Evidence of Professional Development/Technical Assistance for Successful Implementation: This rating refers to evidence that the model provider offers comprehensive training 
opportunities and supporting materials, ensures that professional development effectively supports full model implementation, and develops the school’s internal capacity to provide professional 
development. 



^ = Very Strong ^ = Moderately Strong 3 = Moderate Q = Limited 0 = Zero Q = Negative (0 = No Rating 
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